Yale NLP Lab - Publications

2025

"On Evaluating LLM Alignment by Evaluating LLMs as Judges"
Yixin Liu, Pengfei Liu, Arman Cohan.
NeurIPS 2025 |
evaluation alignment
"DyFlow: Dynamic Workflow Framework for Agentic Reasoning"
Yanbo Wang, Zixiang Xu, Yue Huang, Xiangqi Wang, Zirui Song, Lang Gao, Chenxi Wang, Xiangru Tang, Yue Zhao, Arman Cohan, Xiangliang Zhang, Xiuying Chen.
NeurIPS 2025 |
agents reasoning
"Measuring what Matters: Construct Validity in Large Language Model Benchmarks"
Andrew M. Bean, Ryan Othniel Kearns, Angelika Romanou, Franziska Sofia Hafner, Harry Mayne, Jan Batzner, Negar Foroutan Eghlidi, Chris Schmitz, Karolina Korgul, Hunar Batra, Oishi Deb, Emma Beharry, Cornelius Emde, Thomas Foster, Anna Gausen, María Grandury, Sophia Simeng Han, Valentin Hofmann, Lujain Ibrahim, Hazel Kim, Hannah Rose Kirk, Fangru Lin, Gabrielle Liu, Lennart Luettgau, Jabez Magomere, Jonathan Rystrøm, Anna Sotnikova, Yushi Yang, Yilun Zhao, Adel Bibi, Antoine Bosselut, Ronald Clark, Arman Cohan, Jakob Foerster, Yarin Gal, Scott Hale, Deborah Raji, Christopher Summerfield, Philip Torr, Cozmin Ududec, Luc Rocher, Adam Mahdi.
NeurIPS 2025 |
evaluation
"SciArena: An Open Evaluation Platform for Foundation Models in Scientific Literature Tasks"
Yilun Zhao, Kaiyan Zhang, Tiansheng Hu, Sihong Wu, Ronan Le Bras, Taira Anderson, Jonathan Bragg, Joseph Chee Chang, Jesse Dodge, Matt Latzke, Yixin Liu, Charles McGrady, Xiangru Tang, Zihang Wang, Chen Zhao, Hannaneh Hajishirzi, Doug Downey, Arman Cohan.
NeurIPS 2025 |
evaluation foundation models
"CourtReasoner: Can LLM Agents Reason Like Judges?"
Simeng Han, Yoshiki Takashima, Shannon Zejiang Shen, Chen Liu, Yixin Liu, Roque K. Thuo, Sonia Knowlton, Ruzica Piskac, Scott J. Shapiro, Arman Cohan.
EMNLP 2025 |
evaluation specialized domains
"Diffusion vs. Autoregressive Language Models: A Text Embedding Perspective"
Siyue Zhang, Yilun Zhao, Liyuan Geng, Arman Cohan, Anh Tuan Luu, Chen Zhao.
EMNLP 2025 |
retrieval evaluation
"Judging with Many Minds: Do More Perspectives Mean Less Prejudice?"
Chiyu Ma, Enpei Zhang, Yilun Zhao, Wenjun Liu, Yaning Jia, Peijun Qing, Lin Shi, Arman Cohan, Yujun Yan, Soroush Vosoughi.
EMNLP 2025 |
language modelings evaluation reasoning
"MetaFaith: Faithful Natural Language Uncertainty Expression in LLMs"
Gabrielle Kaili-May Liu, Gal Yona, Avi Caciularu, Idan Szpektor, Tim G. J. Rudner, Arman Cohan.
EMNLP 2025 |
evaluation safety and alignment llm agents
"SciRIFF: A Resource to Enhance Language Model Instruction-Following over Scientific Literature"
David Wadden, Kejian Shi, Jacob Daniel Morrison, Aakanksha Naik, Shruti Singh, Nitzan Barzilay, Kyle Lo, Tom Hope, Luca Soldaini, Shannon Zejiang Shen, Doug Downey, Hanna Hajishirzi, Arman Cohan
EMNLP 2025 |
post-training evaluation safety and alignment
"Table-R1: Inference-Time Scaling for Table Reasoning"
Zheyuan Yang, Lyuhao Chen, Arman Cohan, Yilun Zhao
EMNLP 2025 |
AI for science post-training
"MCTS-RAG: Enhancing Retrieval-Augmented Generation with Monte Carlo Tree Search"
Yunhai Hu, Yilun Zhao, Chen Zhao, Arman Cohan
EMNLP 2025, Findings |
reasoning post-training efficiency and scalability of LLMs
"MSRS: Evaluating Multi-Source Retrieval-Augmented Generation"
Rohan Phanse, Yijie Zhou, Kejian Shi, Wencai Zhang, Yixin Liu, Yilun Zhao, Arman Cohan.
COLM 2025 |
evaluation reasoning
"One-shot Optimized Steering Vectors Mediate Safety-relevant Behaviors in LLMs"
Jacob Dunefsky, Arman Cohan.
COLM 2025 |
evaluation reasoning language modelings
"Understanding Reference Policies in Direct Preference Optimization"
Yixin Liu, Pengfei Liu, Arman Cohan.
NAACL 2025, findings |
reinforcement learning preference learning document understanding
"TOMATO: Assessing Visual Temporal Reasoning Capabilities in Multimodal Foundation Models"
Ziyao Shangguan, Chuhan Li, Yuxuan Ding, Yanan Zheng, Yilun Zhao, Tesca Fitzgerald, Arman Cohan.
ICLR 2025 |
multimodal vision evaluation
"ReIFE: Re-evaluating Instruction-Following Evaluation"
Yixin Liu, Kejian Shi, Alexander Fabbri, Yilun Zhao, Peifeng Wang, Chien-Sheng Wu, Shafiq Joty, Arman Cohan.
NAACL 2025 |
evaluation foundation models document understanding
"FollowIR: Evaluating and Teaching Information Retrieval Models to Follow Instructions"
Orion Weller, Benjamin Chang, Sean MacAvaney, Kyle Lo, Arman Cohan, Benjamin Van Durme, Dawn Lawrie, Luca Soldaini.
NAACL 2025 |
evaluation training methods document understanding
"ChemAgent: Self-updating Library in Large Language Models Improves Chemical Reasoning"
Xiangru Tang, Tianyu Hu, Muyang Ye, Yanjun Shao, Xunjian Yin, Siru Ouyang, Wangchunshu Zhou, Pan Lu, Zhuosheng Zhang, Yilun Zhao, Arman Cohan, Mark Gerstein.
ICLR 2025 |
specialized domains document understanding LLM agents
"MMVU: Measuring Expert-Level Multi-Discipline Video Understanding"
Yilun Zhao, Lujing Xie, Haowei Zhang, Guo Gan, Yitao Long, Zhiyuan Hu, Tongyan Hu, Weiyuan Chen, Chuhan Li, Junyang Song, Zhijian Xu, Chengye Wang, Weifeng Pan, Ziyao Shangguan, Xiangru Tang, Zhenwen Liang, Yixin Liu, Chen Zhao, Arman Cohan.
Preprint |
foundation models evaluation specialized domains
"RouterRetriever: Exploring the Benefits of Routing over Multiple Expert Embedding Models"
Hyunji Lee, Luca Soldaini, Arman Cohan, Minjoon Seo, Kyle Lo.
AAAI 2025 |
document understanding evaluation specialized domains

2024

"FOLIO: Natural Language Reasoning with First-Order Logic"
Simeng Han, Hailey Schoelkopf, Yilun Zhao, Zhenting Qi, Martin Riddell, Luke Benson, Lucy Sun, E. Zubova, Yujie Qiao, Matthew Burtell, David Peng, Jonathan Fan, Yixin Liu, Brian Wong, Malcolm Sailor, Ansong Ni, Linyong Nan, Jungo Kasai, Tao Yu, Rui Zhang, Shafiq R. Joty, Alexander R. Fabbri, Wojciech Kryscinski, Xi Victoria Lin, Caiming Xiong, Dragomir R. Radev.
EMNLP 2024 |
- evaluation - training methods - interpretability
"Bayesian Calibration of Win Rate Estimation with LLM Evaluators"
Yicheng Gao, Gonghan Xu, Zhe Wang, Arman Cohan.
EMNLP 2024 |
evaluation foundation models training methods
"FinDVer: Explainable Claim Verification over Long and Hybrid-content Financial Documents"
Yilun Zhao, Yitao Long, Tintin Jiang, Chengye Wang, Weiyuan Chen, Hongjun Liu, Xiangru Tang, Yiming Zhang, Chen Zhao, Arman Cohan.
EMNLP 2024 |
document understanding evaluation specialized domains
"SciDQA: A Deep Reading Comprehension Dataset over Scientific Papers"
Shruti Singh, Nandan Sarkar, Arman Cohan.
EMNLP 2024 |
document understanding evaluation foundation models
"P-FOLIO: Evaluating and Improving Logical Reasoning with Abundant Human-Written Reasoning Chains"
Simeng Han, Aaron Yu, Rui Shen, Zhenting Qi, Martin Riddell, Wenfei Zhou, Yujie Qiao, Yilun Zhao, Semih Yavuz, Ye Liu, Shafiq Joty, Yingbo Zhou, Caiming Xiong, Dragomir R. Radev, Rex Ying, Arman Cohan.
EMNLP 2024 Findings |
evaluation training methods interpretability
"Calibrating Long-form Generations from Large Language Models"
Yukun Huang, Yixin Liu, Raghuveer Thirukovalluru, Arman Cohan, Bhuwan Dhingra.
EMNLP 2024 Findings |
evaluation training methods foundation models
"M3SciQA: A Multi-Modal Multi-Document Scientific QA Benchmark for Evaluating Foundation Models"
Chuhan Li, Ziyao Shangguan, Yilun Zhao, Deyuan Li, Yixin Liu, Arman Cohan.
EMNLP 2024 Findings |
foundation models evaluation document understanding
"OMG-QA: Building Open-Domain Multi-Modal Generative Question Answering Systems"
Linyong Nan, Weining Fang, Aylin Rasteh, Pouya Lahabi, Weijin Zou, Yilun Zhao, Arman Cohan.
EMNLP 2024 Industry Track |
- evaluation - document understanding - specialized domains
"MIMIR: A Customizable Agent Tuning Platform for Enhanced Scientific Applications"
Xiangru Tang, Chunyuan Deng, Hanminwang Hanminwang, Haoran Wang, Yilun Zhao, Wenqi Shi, Yi Fung, Wangchunshu Zhou, Jiannan Cao, Heng Ji, Arman Cohan, Mark Gerstein.
EMNLP 2024 Demo |
- LLM agents - training methods - specialized domains
"OpenT2T: An Open-Source Toolkit for Table-to-Text Generation"
Haowei Zhang, Shengyun Si, Yilun Zhao, Lujing Xie, Zhijian Xu, Lyuhao Chen, Linyong Nan, Pengcheng Wang, Xiangru Tang, Arman Cohan.
EMNLP 2024 Demo |
document understanding evaluation foundation models
"TAIL: A Toolkit for Automatic and Realistic Long-Context Large Language Model Evaluation"
Gefei Gu, Yilun Zhao, Ruoxi Ning, Yanan Zheng, Arman Cohan.
EMNLP 2024 Demo |
evaluation foundation models specialized domains
"Observable Propagation: Uncovering Feature Vectors in Transformers"
Jacob Dunefsky, Arman Cohan
ICML 2024 |
interpretability
"NExT: Teaching Large Language Models to Reason about Code Execution"
Ansong Ni, Miltiadis Allamanis, Arman Cohan, Yinlin Deng, Kensen Shi, Charles Sutton, Pengcheng Yin
ICML 2024 |
training methods code generation interpretability
"On Learning to Summarize with Large Language Models as References"
Yixin Liu, Alexander R. Fabbri, Pengfei Liu, Dragomir R. Radev, Arman Cohan.
NAACL 2024 |
foundation models
"Investigating Data Contamination in Modern Benchmarks for Large Language Models"
Chunyuan Deng, Yilun Zhao, Xiangru Tang, Mark B. Gerstein, Arman Cohan.
NAACL 2024 |
evaluation foundation models interpretability
"Struc-Bench: Are Large Language Models Really Good at Generating Complex Structured Data?"
Xiangru Tang, Yiming Zong, Jason Phang, Yilun Zhao, Wangchunshu Zhou, Arman Cohan, Mark Gerstein.
NAACL 2024 |
foundation models evaluation training methods
"Struc-Bench: Are Large Language Models Good at Generating Complex Structured Tabular Data?"
Xiangru Tang, Yiming Zong, Jason Phang, Yilun Zhao, Wangchunshu Zhou, Arman Cohan, Mark Gerstein.
NAACL 2024 |
- evaluation - training methods - document understanding
"On Evaluating the Integration of Reasoning and Action in LLM Agents with Database Question Answering"
Linyong Nan, Ellen Zhang, Weijin Zou, Yilun Zhao, Wenfei Zhou, Arman Cohan.
NAACL 2024 Findings |
LLM agents evaluation document understanding
"Benchmarking Generation and Evaluation Capabilities of Large Language Models for Instruction Controllable Summarization"
Yixin Liu, A. R. Fabbri, Jiawen Chen, Yilun Zhao, Simeng Han, Shafiq R. Joty, Pengfei Liu, Dragomir R. Radev, Chien-Sheng Wu, Arman Cohan.
NAACL 2024 Findings |
evaluation training methods document understanding
"Quantifying Contamination in Evaluating Code Generation Capabilities of Language Models"
Martin Riddell, Ansong Ni, Arman Cohan.
ACL 2024 |
evaluation code generation
"OLMo: Accelerating the Science of Language Models"
Dirk Groeneveld, Iz Beltagy, Pete Walsh, Akshita Bhagia, Rodney Kinney, Oyvind Tafjord, A. Jha, Hamish Ivison, Ian Magnusson, Yizhong Wang, Shane Arora, David Atkinson, Russell Authur, Khyathi Raghavi Chandu, Arman Cohan, Jennifer Dumas, Yanai Elazar, Yuling Gu, Jack Hessel, Tushar Khot, William Merrill, Jacob Daniel Morrison, Niklas Muennighoff, Aakanksha Naik, Crystal Nam, Matthew E. Peters, Valentina Pyatkin, Abhilasha Ravichander, Dustin Schwenk, Saurabh Shah, Will Smith, Emma Strubell, Nishant Subramani, Mitchell Wortsman, Pradeep Dasigi, Nathan Lambert, Kyle Richardson, Luke Zettlemoyer, Jesse Dodge, Kyle Lo, Luca Soldaini, Noah A. Smith, Hanna Hajishirzi.
ACL 2024 |
foundation models
"TaPERA: Enhancing Faithfulness and Interpretability in Long-Form Table QA by Content Planning and Execution-based Reasoning"
Yilun Zhao, Lyuhau Chen, Arman Cohan, Chen Zhao
ACL 2024 |
reasoning evaluation document understanding
FinanceMath: A Knowledge-Intensive Math Reasoning Dataset in Finance Domains
Yilun Zhao, Hongjun Liu, Yitao Long, Rui Zhang, Chen Zhao, Arman Cohan
ACL 2024 |
dataset reasoning evaluation document understanding specialized domains
"DocMath-Eval: Evaluating Math Reasoning Capabilities of LLMs in Understanding Financial Documents"
Yilun Zhao, Yitao Long, Hongjun Liu, Ryo Kamoi, Linyong Nan, Lyuhao Chen, Yixin Liu, Xiangru Tang, Rui Zhang, Arman Cohan.
ACL 2024 |
- evaluation - document understanding - specialized domains
"Rethinking Efficient Multilingual Text Summarization Meta-Evaluation"
Rilyn Han, Jiawen Chen, Yixin Liu, Arman Cohan.
ACL 2024 |
- evaluation - document understanding - specialized domains
"Unveiling the Spectrum of Data Contamination in Language Models: A Survey from Detection to Remediation"
Chunyuan Deng, Yilun Zhao, Yuzhao Heng, Yitong Li, Jiannan Cao, Xiangru Tang, Arman Cohan.
ACL 2024 |
evaluation training methods foundation models
"Unveiling the Spectrum of Data Contamination in Language Models: A Survey from Detection to Remediation"
Chunyuan Deng, Yilun Zhao, Yuzhao Heng, Yitong Li, Jiannan Cao, Xiangru Tang, Arman Cohan
ACL 2024 findings |
evaluation contamination survey
"MedAgents: Large Language Models as Collaborators for Zero-shot Medical Reasoning"
Xiangru Tang, Anni Zou, Zhuosheng Zhang, Ziming Li, Yilun Zhao, Xingyao Zhang, Arman Cohan, Mark Gerstein
ACL 2024 findings |
LM agents specialized domains reasoning
"On the Benefits of Fine-Grained Loss Truncation: A Case Study on Factuality in Summarization"
Lorenzo Jaime Yu Flores, Arman Cohan.
EACL 2024 |
evaluation training methods document understanding
"Step-Back Profiling: Distilling User History for Personalized Scientific Writing"
Xiangru Tang, Xingyao Zhang, Yanjun Shao, Jie Wu, Yilun Zhao, Arman Cohan, Ming Gong, Dongmei Zhang, Mark Gerstein
AI4Science workshop @IJCAI 2024 |
personalization writing generation
"Evaluating LLMs at Detecting Errors in LLM Responses"
Ryo Kamoi, Sarkar Snigdha Sarathi Das, Renze Lou, Jihyun Janice Ahn, Yilun Zhao, Xiaoxin Lu, Nan Zhang, Yusen Zhang, Ranran Haoran Zhang, Sujeeth Reddy Vummanthala, Salika Dave, Shaobo Qin, Arman Cohan, Wenpeng Yin, Rui Zhang.
COLM 2024 |
evaluation foundation models interpretability
"L2CEval: Evaluating Language-to-Code Generation Capabilities of Large Language Models"
Ansong Ni, Pengcheng Yin, Yilun Zhao, Martin Riddell, Troy Feng, Rui Shen, Stephen Yin, Ye Liu, Semih Yavuz, Caiming Xiong, Shafiq R. Joty, Yingbo Zhou, Dragomir R. Radev, Arman Cohan.
TACL |
evaluation code generation
"Prioritizing Safeguarding Over Autonomy: Risks of LLM Agents for Science"
Xiangru Tang, Qiao Jin, Kunlun Zhu, Tongxin Yuan, Yichi Zhang, Wangchunshu Zhou, Meng Qu, Yilun Zhao, Jian Tang, Zhuosheng Zhang, Arman Cohan, Zhiyong Lu, Mark B. Gerstein.
Preprint |
LLM agents specialized domains evaluation
"SciRIFF: A Resource to Enhance Language Model Instruction-Following over Scientific Literature"
David Wadden, Kejian Shi, Jacob Daniel Morrison, Aakanksha Naik, Shruti Singh, Nitzan Barzilay, Kyle Lo, Tom Hope, Luca Soldaini, Shannon Zejiang Shen, Doug Downey, Hanna Hajishirzi, Arman Cohan.
Preprint |
- foundation models - evaluation - document understanding
"COMAL: A Convergent Meta-Algorithm for Aligning LLMs with General Preferences"
Yixin Liu, Argyris Oikonomou, Weiqiang Zheng, Yang Cai, Arman Cohan.
Preprint |
foundation models evaluation training methods
"MetaMath: Integrating Natural Language and Code for Enhanced Mathematical Reasoning in Large Language Models"
Xuyuan Xiong, Simeng Han, Ziyue Zhou, Arman Cohan.
Preprint |
code generation document understanding LLM agents

2023

"A Question Answering Framework for Decontextualizing User-facing Snippets from Scientific Documents"
Benjamin Newman, Luca Soldaini, Raymond Fok, Arman Cohan, Kyle Lo.
EMNLP 2023 |
document understanding evaluation LLM agents
"QTSumm: A New Benchmark for Query-Focused Table Summarization"
Yilun Zhao, Zhenting Qi, Linyong Nan, Boyu Mi, Yixin Liu, Weijin Zou, Simeng Han, Xiangru Tang, Yumo Xu, Arman Cohan, Dragomir R. Radev.
EMNLP 2023 |
document understanding evaluation foundation models
"Enhancing Text-to-SQL Capabilities of Large Language Models: A Study on Prompt Design Strategies"
Linyong Nan, Yilun Zhao, Weijin Zou, Narutatsu Ri, Jaesung Tae, Ellen Zhang, Arman Cohan, Dragomir R. Radev.
EMNLP 2023 Findings |
- foundation models - training methods - document understanding
"Medical Text Simplification: Optimizing for Readability with Unlikelihood Training and Reranked Beam Search Decoding"
Lorenzo Jaime Yu Flores, Heyuan Huang, Kejian Shi, Sophie Chheang, Arman Cohan.
EMNLP 2023 Findings |
specialized domains document understanding evaluation
"Enhancing Few-shot Text-to-SQL Capabilities of Large Language Models: A Study on Prompt Design Strategies"
Linyong Nan, Yilun Zhao, Weijin Zou, Narutatsu Ri, Jaesung Tae, E. Zhang, Arman Cohan, Dragomir R. Radev.
EMNLP 2023 Findings |
- foundation models - evaluation - document understanding
"Open Domain Multi-document Summarization: A Comprehensive Study of Model Brittleness under Retrieval"
John Giorgi, Luca Soldaini, Bo Wang, Gary Bader, Kyle Lo, Lucy Lu Wang, Arman Cohan.
EMNLP 2023 Findings |
document understanding evaluation training methods
"Investigating Table-to-Text Generation Capabilities of LLMs in Real-World Information Seeking Scenarios"
Yilun Zhao, Haowei Zhang, Shengyun Si, Linyong Nan, Xiangru Tang, Arman Cohan.
EMNLP 2023 Industry Track |
document understanding evaluation foundation models
"Investigating Table-to-Text Generation Capabilities of Large Language Models in Real-World Information Seeking Scenarios"
Yilun Zhao, Haowei Zhang, Shengyun Si, Linyong Nan, Xiangru Tang, Arman Cohan.
EMNLP 2023 industry track |
document understanding LLM agents foundation models
"Peek Across: Improving Multi-Document Modeling via Cross-Document Question-Answering"
Avi Caciularu, Matthew E. Peters, J. Goldberger, Ido Dagan, Arman Cohan.
ACL 2023 |
multi-document modeling evaluation training methods
"OpenRT: An Open-source Framework for Reasoning Over Tabular Data"
Yilun Zhao, Boyu Mi, Zhenting Qi, Linyong Nan, Minghao Guo, Arman Cohan, Dragomir R. Radev.
ACL 2023 Demo |
document understanding evaluation specialized domains
"TESS: Text-to-Text Self-Conditioned Simplex Diffusion"
Rabeeh Karimi Mahabadi, Jaesung Tae, Hamish Ivison, J. Henderson, Iz Beltagy, Matthew E. Peters, Arman Cohan.
EACL 2023 |
training methods document understanding foundation models
"LongEval: Guidelines for Human Evaluation of Faithfulness in Long-form Summarization"
Kalpesh Krishna, Erin Bransom, Bailey Kuehl, Mohit Iyyer, Pradeep Dasigi, Arman Cohan, Kyle Lo.
EACL 2023 |
evaluation document understanding specialized domains
"Aligning Factual Consistency for Clinical Studies Summarization through Reinforcement Learning"
Xiangru Tang, Arman Cohan, M. Gerstein.
Clinical Natural Language Processing Workshop 2023 |
- document understanding - evaluation - training methods
"Natural language processing in radiology: Clinical applications and future directions."
Pratheek S Bobba, A. Sailer, J. Pruneski, Spencer Beck, Ali Mozayan, Sara Mozayan, Jennifer Arango, Arman Cohan, S. Chheang.
Clinical imaging 2023 |
specialized domains document understanding evaluation
"When do Generative Query and Document Expansions Fail? A Comprehensive Study Across Methods, Retrievers, and Datasets"
Orion Weller, Kyle Lo, David Wadden, Dawn J Lawrie, Benjamin Van Durme, Arman Cohan, Luca Soldaini.
Findings 2023 |
evaluation training methods document understanding
"ReFIT: Relevance Feedback from a Reranker during Inference"
R. Reddy, Pradeep Dasigi, Md Arafat Sultan, Arman Cohan, Avirup Sil, Heng Ji, Hannaneh Hajishirzi.
Preprint |
document understanding evaluation training methods
"ODSum: New Benchmarks for Open Domain Multi-Document Summarization"
Yijie Zhou, Kejian Shi, Wencai Zhang, Yixin Liu, Yilun Zhao, Arman Cohan.
Preprint |
document understanding evaluation training methods
"The Semantic Scholar Open Data Platform"
Rodney Michael Kinney, Chloe Anastasiades, Russell Authur, Iz Beltagy, Jonathan Bragg, Alexandra Buraczynski, Isabel Cachola, Stefan Candra, Yoganand Chandrasekhar, Arman Cohan, Miles Crawford, Doug Downey, Jason Dunkelberger, Oren Etzioni, Rob Evans, Sergey Feldman, Joseph Gorney, D. Graham, F.Q. Hu, Regan Huff, Daniel King, Sebastian Kohlmeier, Bailey Kuehl, Michael Langan, Daniel Lin, Haokun Liu, Kyle Lo, Jaron Lochner, Kelsey MacMillan, Tyler Murray, Christopher Newell, Smita R Rao, Shaurya Rohatgi, P. Sayre, Zejiang Shen, Amanpreet Singh, Luca Soldaini, Shivashankar Subramanian, A. Tanaka, Alex D Wade, Linda M. Wagner, Lucy Lu Wang, Christopher Wilhelm, Caroline Wu, Jiangjiang Yang, Angele Zamarron, Madeleine van Zuylen, Daniel S. Weld.
Preprint |
document understanding foundation models evaluation
"ML-Bench: Evaluating Large Language Models for Code Generation in Repository-Level Machine Learning Tasks"
Yuliang Liu, Xiangru Tang, Zefan Cai, Junjie Lu, Yichi Zhang, Yan Shao, Zexuan Deng, Helan Hu, Zengxian Yang, Kaikai An, Ruijun Huang, Shuzheng Si, Sheng Chen, Haozhe Zhao, Zheng Li, Liang Chen, Yiming Zong, Yan Wang, Tianyu Liu, Zhiwei Jiang, Baobao Chang, Yujia Qin, Wangchunshu Zhou, Yilun Zhao, Arman Cohan, Mark B. Gerstein.
Preprint |
code generation evaluation LLM agents

2022

"SciFact-Open: Towards open-domain scientific claim verification"
David Wadden, Kyle Lo, Bailey Kuehl, Arman Cohan, Iz Beltagy, Lucy Lu Wang, Hannaneh Hajishirzi.
EMNLP 2022 |
document understanding evaluation specialized domains
"Open Domain Multi-document Summarization: A Comprehensive Study of Model Brittleness under Retrieval"
John Giorgi, Luca Soldaini, Bo Wang, Gary Bader, Kyle Lo, Lucy Lu Wang, Arman Cohan.
EMNLP 2022 |
- document understanding - evaluation - training methods
"SciRepEval: A Multi-Format Benchmark for Scientific Document Representations"
Amanpreet Singh, Mike D'Arcy, Arman Cohan, Doug Downey, Sergey Feldman.
EMNLP 2022 |
document understanding evaluation training methods
"Generating Scientific Claims for Zero-Shot Scientific Fact Checking"
Dustin Wright, David Wadden, Kyle Lo, Bailey Kuehl, Arman Cohan, Isabelle Augenstein, Lucy Lu Wang.
ACL 2022 |
- document understanding - evaluation - specialized domains
"Improving the Generalizability of Depression Detection by Leveraging Clinical Questionnaires"
Thong Nguyen, Andrew Yates, Ayah Zirikly, Bart Desmet, Arman Cohan.
ACL 2022 |
- interpretability - evaluation - specialized domains
"Zero- and Few-Shot NLP with Pretrained Language Models"
Iz Beltagy, Arman Cohan, Robert Logan IV, Sewon Min, Sameer Singh.
ACL 2022 |
training methods foundation models evaluation
"Revisiting the Gold Standard: Grounding Summarization Evaluation with Robust Human Evaluation"
Yixin Liu, Alexander R. Fabbri, Pengfei Liu, Yilun Zhao, Linyong Nan, Ruilin Han, Simeng Han, Shafiq R. Joty, Chien-Sheng Wu, Caiming Xiong, Dragomir R. Radev.
ACL 2022 |
evaluation foundation models document understanding
"Embedding Recycling for Language Models"
Jon Saad-Falcon, Amanpreet Singh, Luca Soldaini, Mike D'Arcy, Arman Cohan, Doug Downey.
Findings 2022 |
evaluation training methods foundation models
"Overview of the First Shared Task on Multi Perspective Scientific Document Summarization (MuP)"
Arman Cohan, Guy Feigenblat, Tirthankar Ghosal, Michal Shmueli-Scheuer.
SDP 2022 |
document understanding evaluation foundation models
"Overview of the Third Workshop on Scholarly Document Processing"
Arman Cohan, Guy Feigenblat, Dayne Freitag, Tirthankar Ghosal, Drahomira Herrmannova, Petr Knoth, Kyle Lo, Philipp Mayr, Michal Shmueli-Scheuer, Anita de Waard, Lucy Lu Wang.
SDP 2022 |
document understanding evaluation foundation models

2021

"Cross-Document Language Modeling"
Avi Caciularu, Arman Cohan, Iz Beltagy, Matthew E. Peters, Arie Cattan, Ido Dagan.
EMNLP 2021 |
foundation models training methods document understanding
"FLEX: Unifying Evaluation for Few-Shot NLP"
Jonathan Bragg, Arman Cohan, Kyle Lo, Iz Beltagy.
NeurIPS 2021 |
evaluation foundation models training methods
"A Dataset of Information-Seeking Questions and Answers Anchored in Research Papers"
Pradeep Dasigi, Kyle Lo, Iz Beltagy, Arman Cohan, Noah A. Smith, Matt Gardner.
NAACL 2021 |
document understanding evaluation specialized domains
"Beyond Paragraphs: NLP for Long Sequences"
Iz Beltagy, Arman Cohan, Hannaneh Hajishirzi, Sewon Min, Matthew E. Peters.
NAACL 2021 |
document understanding foundation models evaluation
"Long Context Question Answering via Supervised Contrastive Learning"
Avi Caciularu, Ido Dagan, J. Goldberger, Arman Cohan.
NAACL 2021 |
long-context question answering evaluation training methods
"Multi-Vector Models with Textual Guidance for Fine-Grained Scientific Document Similarity"
Sheshera Mysore, Arman Cohan, Tom Hope.
NAACL 2021 |
document understanding evaluation training methods
"MultiCite: Modeling realistic citations requires moving beyond the single-sentence single-label setting"
Anne Lauscher, B. Ko, Bailey Kuehl, Sophie Johnson, David Jurgens, Arman Cohan, Kyle Lo.
NAACL 2021 |
document understanding evaluation specialized domains
"PRIMERA: Pyramid-based Masked Sentence Pre-training for Multi-document Summarization"
Wen Xiao, Iz Beltagy, G. Carenini, Arman Cohan.
ACL 2021 |
foundation models evaluation document understanding
"Simplified Data Wrangling with ir_datasets"
Sean MacAvaney, Andrew Yates, Sergey Feldman, Doug Downey, Arman Cohan, Nazli Goharian.
Annual International ACM SIGIR Conference on Research and Development in Information Retrieval 2021 |
document understanding specialized domains evaluation
"Two Contrasting Data Annotation Paradigms for Subjective NLP Tasks"
David Wadden, Kyle Lo, Lucy Lu Wang, Arman Cohan, Iz Beltagy, Hannaneh Hajishirzi.
NAACL-HLT 2021 |
document understanding evaluation specialized domains
"LongChecker: Improving scientific claim verification by modeling full-abstract context"
David Wadden, Kyle Lo, Lucy Lu Wang, Arman Cohan, Iz Beltagy, Hannaneh Hajishirzi.
Preprint |
document understanding evaluation specialized domains
"Overview of the Second Workshop on Scholarly Document Processing"
Iz Beltagy, Arman Cohan, Guy Feigenblat, Dayne Freitag, Tirthankar Ghosal, Keith B. Hall, Drahomira Herrmannova, Petr Knoth, Kyle Lo, Philipp Mayr, Robert M. Patton, Michal Shmueli-Scheuer, A. Waard, Kuansan Wang, Lucy Lu Wang.
Preprint |
document understanding foundation models evaluation
"PRIMER: Pyramid-based Masked Sentence Pre-training for Multi-document Summarization"
Wen Xiao, Iz Beltagy, G. Carenini, Arman Cohan.
Preprint |
foundation models training methods document understanding
"Utilizing Evidence Spans via Sequence-Level Contrastive Learning for Long-Context Question Answering"
Avi Caciularu, Ido Dagan, J. Goldberger, Arman Cohan.
Preprint |
- training methods - evaluation - document understanding

2020

"Fact or Fiction: Verifying Scientific Claims"
David Wadden, Kyle Lo, Lucy Lu Wang, Shanchuan Lin, Madeleine van Zuylen, Arman Cohan, Hannaneh Hajishirzi.
EMNLP 2020 |
document understanding evaluation specialized domains
"SLEDGE: A Simple Yet Effective Zero-Shot Baseline for Coronavirus Scientific Knowledge Search"
Sean MacAvaney, Arman Cohan, Nazli Goharian.
EMNLP 2020 |
document understanding evaluation specialized domains
"SPECTER: Document-level Representation Learning using Citation-informed Transformers"
Arman Cohan, Sergey Feldman, Iz Beltagy, Doug Downey, Daniel S. Weld.
ACL 2020 |
document understanding evaluation foundation models
"Ranking Significant Discrepancies in Clinical Reports"
Sean MacAvaney, Arman Cohan, Nazli Goharian, Ross W. Filice.
European Conference on Information Retrieval 2020 |
- document understanding - specialized domains - evaluation
"TLDR: Extreme Summarization of Scientific Documents"
Isabel Cachola, Kyle Lo, Arman Cohan, Daniel S. Weld.
Findings 2020 |
document understanding evaluation training methods
"GUIR @ LongSumm 2020: Learning to Generate Long Summaries from Scientific Documents"
Sajad Sotudeh Gharebagh, Arman Cohan, Nazli Goharian.
SDP 2020 |
document understanding evaluation training methods
"On Generating Extended Summaries of Long Documents"
Sajad Sotudeh, Arman Cohan, Nazli Goharian.
SDU@AAAI 2020 |
document understanding evaluation training methods
"ParsiNLU: A Suite of Language Understanding Challenges for Persian"
Daniel Khashabi, Arman Cohan, Siamak Shakeri, Pedram Hosseini, Pouya Pezeshkpour, Malihe Alikhani, Moin Aminnaseri, Marzieh Bitaab, Faeze Brahman, Sarik Ghazarian, Mozhdeh Gheini, Arman Kabiri, Rabeeh Karimi Mahabadi, Omid Memarrast, Ahmadreza Mosallanezhad, Erfan Noury, Shahab Raji, Mohammad Sadegh Rasooli, Sepideh Sadeghi, Erfan Sadeqi Azer, Niloofar Safi Samghabadi, Mahsa Shafaei, Saber Sheybani, Ali Tazarv, Yadollah Yaghoobzadeh.
Transactions of the Association for Computational Linguistics 2020 |
document understanding evaluation specialized domains
"Longformer: The Long-Document Transformer"
Iz Beltagy, Matthew E. Peters, Arman Cohan.
Preprint |
foundation models evaluation document understanding
"SLEDGE: A Simple Yet Effective Baseline for Coronavirus Scientific Knowledge Search"
Sean MacAvaney, Arman Cohan, Nazli Goharian.
Preprint |
document understanding evaluation specialized domains

2019

"Pretrained Language Models for Sequential Sentence Classification"
Arman Cohan, Iz Beltagy, Daniel King, Bhavana Dalvi, Daniel S. Weld.
EMNLP 2019 |
training methods foundation models
"SciBERT: A Pretrained Language Model for Scientific Text"
Iz Beltagy, Kyle Lo, Arman Cohan.
EMNLP 2019 |
foundation models language modeling scientific text
"Structural Scaffolds for Citation Intent Classification in Scientific Publications"
Arman Cohan, Bridger Waleed Ammar, Madeleine van Zuylen, Field Cady.
NAACL 2019 |
training methods scientific text specialized domains
"SUPP.AI: finding evidence for supplement-drug interactions"
Lucy Lu Wang, Oyvind Tafjord, Arman Cohan, Sarthak Jain, Sam Skjonsberg, Carissa Schoenick, Nick Botner, Bridger Waleed Ammar.
ACL 2019 |
specialized domains information extraction
"Extracting evidence of supplement-drug interactions from literature"
Lucy Lu Wang, Oyvind Tafjord, Sarthak Jain, Arman Cohan, Sam Skjonsberg, Carissa Schoenick, Nick Botner, Bridger Waleed Ammar.
BioNLP workshop |
specialized domains information extraction
"CEDR: Contextualized Embeddings for Document Ranking"
Sean MacAvaney, Andrew Yates, Arman Cohan, Nazli Goharian.
SIGIR 2019 |
retrieval representation learning
"Ontology-Aware Clinical Abstractive Summarization"
Sean MacAvaney, Sajad Sotudeh, Arman Cohan, Nazli Goharian, Ish A. Talati, Ross W. Filice.
SIGIR 2019 |
scientific text summarization specialized domains
"Text Summarization and Categorization for Scientific and Health-Related Data"
Arman Cohan.
SIGIR 2019 |
specialized domains summarization
"Contextualized Word Representations for Document Re-Ranking"
Sean MacAvaney, Andrew Yates, Arman Cohan, Nazli Goharian.
Preprint |
retrieval representation learning
"SciBERT: Pretrained Contextualized Embeddings for Scientific Text"
Iz Beltagy, Arman Cohan, Kyle Lo.
Preprint |
foundation models evaluation specialized domains

Note: This publication list is partially automatically derived from Semantic Scholar. While we make efforts to manually verify and correct entries, some errors may still be present.

Filter by category:

Filter by publication type:

Search: