Yale NLP Lab - Publications

2026

"MMSciCode: Real-world Evaluation of Multilingual Multi-Discipline Scientific Research Coding"
Xue Xia, Zheyuan Yang, Arman Cohan, Yilun Zhao.
ACL 2026 |
benchmark code-generation evaluation
"CPTCoder: A Reliable LLM System for Medical Procedure Code Prediction"
Benlu Wang, Ziyao Shangguan, Kyle Tegtmeyer, Zhenyu Zhang, Sophie Chheang, Arman Cohan.
ACL 2026 Demo |
code-generation clinical specialized domains
"SciMDR: Advancing Scientific Multimodal Document Reasoning"
Ziyu Chen, Yilun Zhao, Chengye Wang, Rilyn R. Han, Manasi Patwardhan, Arman Cohan.
ACL 2026 |
benchmark multimodal document understanding
"Rethinking Reasoning-Intensive Retrieval: Evaluating and Advancing Retrievers in Agentic Search Systems"
Yilun Zhao, Jinbiao Wei, Tingyu Song, Siyue Zhang, Chen Zhao, Arman Cohan.
ACL 2026 | Code |
retrieval reasoning agents
"Can AI Be a Good Peer Reviewer? A Survey of Peer Review Process, Evaluation, and the Future"
Sihong Wu, Owen Jiang, Yilun Zhao, Tiansheng Hu, Yiling Ma, Kaiyan Zhang, Manasi Patwardhan, Arman Cohan.
ACL 2026 |
evaluation agents scientific-discovery
"A Survey of Multimodal Mathematical Reasoning: From Perception, Alignment to Reasoning"
Tianyu Yang, Sihong Wu, Yilun Zhao, Zhenwen Liang, Lisen Dai, Chen Zhao, Minhao Cheng, Arman Cohan, Xiangliang Zhang.
ACL 2026 |
multimodal reasoning specialized domains
"RbtAct: Rebuttal as Supervision for Actionable Review Feedback Generation"
Sihong Wu, Yiling Ma, Yilun Zhao, Tiansheng Hu, Owen Jiang, Manasi Patwardhan, Arman Cohan.
ACL Findings 2026 |
evaluation scientific-discovery
"Anchor: Branch-Point Data Generation for GUI Agents"
Jinbiao Wei, Yilun Zhao, Kangqi Ni, Arman Cohan.
ACL 2026 | Code |
agents training methods code-generation
"MultiFinBen: Benchmarking Large Language Models for Multilingual and Multimodal Financial Application"
Xueqing Peng, Lingfei Qian, Yan Wang, Ruoyu Xiang, Yueru He, Yang Ren, Mingyang Jiang, Vincent Jim Zhang, Yuqing Guo, Jeff Zhao, Huan He, Yi Han, Yun Feng, Yuechen Jiang, Yupeng Cao, Haohang Li, Yangyang Yu, Xiaoyu Wang, Penglei Gao, Shengyuan Lin, Keyi Wang, Shanshan Yang, Yilun Zhao, Zhiwei Liu, Peng Lu, Jerry Huang, Suyuchen Wang, Triantafillos Papadopoulos, Polydoros Giannouris, Efstathia Soufleri, Nuo Chen, Zhiyang Deng, Heming Fu, Yijia Zhao, Mingquan Lin, Meikang Qiu, Kaleb E Smith, Arman Cohan, Xiao-Yang Liu, Jimin Huang, Guojun Xiong, Alejandro Lopez-Lira, Xi Chen, Junichi Tsujii, Jian-Yun Nie, Sophia Ananiadou, Qianqian Xie.
ACL 2026 |
benchmark multimodal specialized domains
"A Survey on Evaluation of LLM-based Agents"
Asaf Yehudai, Lilach Eden, Alan Li, Guy Uziel, Yilun Zhao, Roy Bar-Haim, Arman Cohan, Michal Shmueli-Scheuer.
ACL Findings 2026 |
evaluation agents
"Reinforcement Learning with Metacognitive Feedback Elicits Faithful Uncertainty Expression in LLMs"
Gabrielle Kaili-May Liu, Avi Caciularu, Gal Yona, Idan Szpektor, Arman Cohan.
COLM 2026 | Code |
reinforcement learning evaluation safety and alignment
"Rethinking Reward Supervision: Rubric-Conditioned Self-Distillation"
Siyi Gu, Jialin Chen, Sophia Zhou, Arman Cohan, Rex Ying.
COLM 2026 | Code |
reinforcement learning post-training alignment
"REVERE: Reflective Evolving Research Engineer for Scientific Workflows"
Balaji Dinesh Gangireddi, Aniketh Garikaparthi, Manasi Patwardhan, Arman Cohan.
COLM 2026 |
agents code-generation scientific-discovery
"VideoKR: Towards Knowledge- and Reasoning-Intensive Video Understanding" 🏆 Spotlight
Lin Fu, Zheyuan Yang, Yang Wang, Tingyu Song, Arman Cohan, Yilun Zhao.
ICML 2026 | Code |
multimodal reasoning evaluation
"Demystifying Scientific Problem-Solving in LLMs by Probing Knowledge and Reasoning"
Alan Li, Yixin Liu, Arpan Sarkar, Doug Downey, Arman Cohan.
ICML 2026 | Code |
evaluation reasoning scientific-discovery
"QEDBENCH: Quantifying the Alignment Gap in Automated Evaluation of University-Level Mathematical Proofs"
Santiago Gonzalez, Alireza Amiri Bavandpour, Peter Ye, Edward Zhang, Ruslans Aleksejevs, Todor Antić, Polina Baron, Sujeet Bhalerao, Shubhrajit Bhattacharya, Zachary Burton, John Byrne, Hyungjun Choi, Nujhat Ahmed Disha, Koppany István Encz, Yuchen Fang, Robert Joseph George, Ebrahim Ghorbani, Alan Goldfarb, Jing Guo, Meghal Gupta, Stefano Huber, Annika Kanckos, Minjung Kang, Hyun Jong Kim, Dino Lorenzini, Levi Lorenzo, Tianyi Mao, Giovanni Marzenta, Ariane M. Masuda, Lukas Mauth, Ana Mickovic, Andres Miniguano-Trujillo, Antoine Moulin, Wenqi Ni, Tomos Parry, Kevin Ren, Hossein Roodbarani, Mathieu Rundström, Manjil Saikia, Detchat Samart, Rebecca Steiner, Connor Stewart, Dhara Thakkar, Jeffrey Tse, Vasiliki Velona, Yunhai Xiang, Sibel Yalçın, Jun Yan, Ji Zeng, Arman Cohan, Quanquan C. Liu.
ICML 2026 | Code |
benchmark evaluation specialized domains
"Can Scientific Claims Be Removed from Large Language Models? A Systematic Evaluation of Claim-Level Unlearning"
Snigdha Paul, Manasi Patwardhan, Arman Cohan.
Preprint |
training methods scientific-discovery safety and alignment
"Metacognition in LLMs: Foundations, Progress, and Opportunities"
Gabrielle Kaili-May Liu, Areeb Gani, Jacqueline Lu, Jordan Thomas, Mark Steyvers, Arman Cohan.
Preprint | Code |
interpretability evaluation foundation models
"Measuring the Gap Between Human and LLM Research Ideas"
Ziyu Chen, Yilun Zhao, Arman Cohan.
Preprint |
evaluation scientific-discovery foundation models
"GUI vs. CLI: Execution Bottlenecks in Screen-Only and Skill-Mediated Computer-Use Agents"
Xiao Zhou, Siyue Zhang, Yilun Zhao, Jinbiao Wei, Tingyu Song, Arman Cohan, Chen Zhao.
Preprint |
agents evaluation
"Every Eval Ever: A Unifying Schema and Community Repository for AI Evaluation Results"
Jan Batzner, Sree Harsha Nelaturu, Damian Stachura, Anastassia Kornilova, Jon Crall, Tommaso Cerruti, Yanan Long, Yifan Mai, Sanchit Ahuja, Asaf Yehudai, Marek Šuppa, John P. Lalor, Oluwagbemike Olowe, Jatin Ganhotra, Brian H. Hu, Eliya Habba, Andrew M. Bean, Chang Liu, Sander Land, Steven Dillmann, Aniketh Garikaparthi, Elron Bandel, Saki Imai, James Edgell, Wm. Matthew Kennedy, Jenny Chim, Patrick Meusling, Asteria Kaeberlein, Venkata Ramachandra Karthik Chundi, Manasi Patwardhan, Martin Ku, Austin Meek, Leon Knauer, Brian Wingenroth, Srishti Yadav, Usman Gohar, Felix Friedrich, Michelle Lin, Jennifer Mickel, Arman Cohan, Stella Biderman, Irene Solaiman, Zeerak Talat, Anka Reuel, Mubashara Akhtar, Gjergji Kasneci, Avijit Ghosh, Leshem Choshen.
Preprint |
evaluation benchmark
"Benchmarking AI Agents for Addressing Scientific Challenges Across Scales"
Tianyu Liu, Allen Xin Wang, Antonia Panescu, Lisa Xinyi Chen, Wenxin Long, Xinyu Wei, Yueqian Jing, Ziyao Zeng, Jihang Chen, Sihan Jiang, Ziqing Wang, Siyi Gu, Siyu Chen, Xinyang Hu, Haoran Shao, Leqi Xu, Wangjie Zheng, Zhiyuan Cao, Ada Fang, Botao Yu, Kunyang Sun, Rex Ying, Arman Cohan, Qingyu Chen, Lingzhou Xue, Kaize Ding, Yuanqi Du, Wengong Jin, Zhuoran Yang, Marinka Zitnik, James Zou, Hua Xu, Hongyu Zhao.
Preprint |
benchmark agents AI for science
"Quantifying Faithful Confidence Expression in Large Reasoning Models"
Areeb Gani, Asal Meskin, Gabrielle Kaili-May Liu, Arman Cohan.
Preprint | Code |
evaluation interpretability reasoning
"Can LLMs Use Linguistic Uncertainty Markers to Reliably Reflect Intrinsic Confidence?"
Gabrielle Kaili-May Liu, Arman Cohan.
Preprint | Code |
evaluation interpretability
"OpenComputer: Verifiable Software Worlds for Computer-Use Agents"
Jinbiao Wei, Qianran Ma, Yilun Zhao, Xiao Zhou, Kangqi Ni, Guo Gan, Arman Cohan.
Preprint |
agents benchmark code-generation
"Time to REFLECT: Can We Trust LLM Judges for Evidence-based Research Agents?"
Leyao Wang, Yanan He, Peng Chen, Asaf Yehudai, Yixin Liu, Rex Ying, Michal Shmueli-Scheuer, Arman Cohan.
Preprint |
evaluation agents scientific-discovery
"Herculean: An Agentic Benchmark for Financial Intelligence"
Xueqing Peng, Zhuohan Xie, Yupeng Cao, Haohang Li, Lingfei Qian, Yan Wang, Vincent Jim Zhang, Huan He, Xuguang Ai, Linhai Ma, Ruoyu Xiang, Yueru He, Yi Han, Shuyao Wang, Yuqing Guo, Mingyang Jiang, Yilun Zhao, Youzhong Dong, Xiaoyu Wang, Yankai Chen, Ye Yuan, Qiyuan Zhang, Fuyuan Lyu, Haolun Wu, Yonghan Yang, Zichen Zhao, Yuyang Dai, Fan Zhang, Rania Elbadry, Ayesha Gull, Muhammad Usman Safder, Nuo Chen, Fengbin Zhu, Tianshi Cai, Zimu Wang, Polydoros Giannouris, Yuechen Jiang, Zhiwei Liu, Mohsinul Kabir, Yuyan Wang, Yixiang Zheng, Yangyang Yu, Weijin Liu, Wenbo Cao, Anke Xu, Peng Lu, Jerry Huang, Mingquan Lin, Prayag Tiwari, Yijia Zhao, Víctor Gutiérrez-Basulto, Xiao-Yang Liu, Kaleb E Smith, Jiahuan Pei, Arman Cohan, Jimin Huang, Yuehua Tang, Alejandro Lopez-Lira, Xi Chen, Xue Liu, Junichi Tsujii, Jian-Yun Nie, Sophia Ananiadou.
Preprint |
agents benchmark specialized domains
"Make Each Token Count: Towards Improving Long-Context Performance with KV Cache Eviction"
Ngoc Bui, Hieu Trung Nguyen, Arman Cohan, Rex Ying.
Preprint |
efficiency and scalability of LLMs foundation models
"Step-level Optimization for Efficient Computer-use Agents"
Jinbiao Wei, Kangqi Ni, Yilun Zhao, Guo Gan, Arman Cohan.
Preprint | Code |
agents training methods efficiency and scalability of LLMs
"Examining Reasoning LLMs-as-Judges in Non-Verifiable LLM Post-Training"
Yixin Liu, Yue Yu, DiJia Su, Sid Wang, Xuewei Wang, Song Jiang, Bo Liu, Arman Cohan, Yuandong Tian, Zhengxing Chen.
Preprint |
evaluation reasoning alignment
"SAGE: Benchmarking and Improving Retrieval for Deep Research Agents"
Tiansheng Hu, Yilun Zhao, Canyu Zhang, Arman Cohan, Chen Zhao.
Preprint |
retrieval agents benchmark
"Patient-Similarity Cohort Reasoning in Clinical Text-to-SQL"
Yifei Shen, Yilun Zhao, Justice Ou, Tinglin Huang, Arman Cohan.
EACL 2026 |
clinical reasoning
"SciRAG: Adaptive, Citation-Aware, and Outline-Guided Retrieval and Synthesis for Scientific Literature"
Hang Ding, Yilun Zhao, Tiansheng Hu, Zihang Wang, Manasi Patwardhan, Arman Cohan.
EACL 2026 |
retrieval rag
"Investigating Retrieval-Augmented Generation Systems on Unanswerable, Uncheatable, Realistic, Multi-hop Queries"
Gabrielle Kaili-May Liu, Bryan Li, Arman Cohan, William Gantt Walden, Eugene Yang.
ECIR 2026 |
retrieval rag evaluation
"ResearchGym: Evaluating Language Model Agents on Real-World AI Research"
Aniketh Garikaparthi, Manasi Patwardhan, Arman Cohan.
ICLR 2026 Workshop |
agent benchmark
"References Improve LLM Alignment in Non-Verifiable Domains"
Kejian Shi, Yixin Liu, Peifeng Wang, Alexander R. Fabbri, Shafiq Joty, Arman Cohan.
ICLR 2026 | Code |
evaluation alignment
"MRMR: A Realistic and Expert-Level Multidisciplinary Benchmark for Reasoning-Intensive Multimodal Retrieval"
Siyue Zhang, Yuan Gao, Xiao Zhou, Yilun Zhao, Tingyu Song, Arman Cohan, Anh Tuan Luu, Chen Zhao.
ICLR 2026 | Code |
benchmark retrieval multimodal
"COMAL: A Convergent Meta-Algorithm for Aligning LLMs with General Preferences"
Yixin Liu, Argyris Oikonomou, Weiqiang Zheng, Yang Cai, Arman Cohan.
ICLR 2026 | Code |
alignment
"Medical thinking with multiple images"
Zonghai Yao, Benlu Wang, Yifan Zhang, Junda Wang, Iris Xia, Zhipeng Tang, Shuo Han, Feiyun Ouyang, Zhichao Yang, Arman Cohan, Hong Yu.
ICLR 2026 | Code |
benchmark multimodal specialized domains

2025

"On Evaluating LLM Alignment by Evaluating LLMs as Judges"
Yixin Liu, Pengfei Liu, Arman Cohan.
NeurIPS 2025 | Code |
evaluation alignment
"DyFlow: Dynamic Workflow Framework for Agentic Reasoning"
Yanbo Wang, Zixiang Xu, Yue Huang, Xiangqi Wang, Zirui Song, Lang Gao, Chenxi Wang, Xiangru Tang, Yue Zhao, Arman Cohan, Xiangliang Zhang, Xiuying Chen.
NeurIPS 2025 |
agents reasoning
"Measuring what Matters: Construct Validity in Large Language Model Benchmarks"
Andrew M. Bean, Ryan Othniel Kearns, Angelika Romanou, Franziska Sofia Hafner, Harry Mayne, Jan Batzner, Negar Foroutan Eghlidi, Chris Schmitz, Karolina Korgul, Hunar Batra, Oishi Deb, Emma Beharry, Cornelius Emde, Thomas Foster, Anna Gausen, María Grandury, Sophia Simeng Han, Valentin Hofmann, Lujain Ibrahim, Hazel Kim, Hannah Rose Kirk, Fangru Lin, Gabrielle Liu, Lennart Luettgau, Jabez Magomere, Jonathan Rystrøm, Anna Sotnikova, Yushi Yang, Yilun Zhao, Adel Bibi, Antoine Bosselut, Ronald Clark, Arman Cohan, Jakob Foerster, Yarin Gal, Scott Hale, Deborah Raji, Christopher Summerfield, Philip Torr, Cozmin Ududec, Luc Rocher, Adam Mahdi.
NeurIPS 2025 |
evaluation
"SciArena: An Open Evaluation Platform for Foundation Models in Scientific Literature Tasks"
Yilun Zhao, Kaiyan Zhang, Tiansheng Hu, Sihong Wu, Ronan Le Bras, Taira Anderson, Jonathan Bragg, Joseph Chee Chang, Jesse Dodge, Matt Latzke, Yixin Liu, Charles McGrady, Xiangru Tang, Zihang Wang, Chen Zhao, Hannaneh Hajishirzi, Doug Downey, Arman Cohan.
NeurIPS 2025 | Code |
evaluation foundation models
"CourtReasoner: Can LLM Agents Reason Like Judges?"
Simeng Han, Yoshiki Takashima, Shannon Zejiang Shen, Chen Liu, Yixin Liu, Roque K. Thuo, Sonia Knowlton, Ruzica Piskac, Scott J. Shapiro, Arman Cohan.
EMNLP 2025 | Code |
evaluation specialized domains
"Diffusion vs. Autoregressive Language Models: A Text Embedding Perspective"
Siyue Zhang, Yilun Zhao, Liyuan Geng, Arman Cohan, Anh Tuan Luu, Chen Zhao.
EMNLP 2025 |
retrieval evaluation
"Judging with Many Minds: Do More Perspectives Mean Less Prejudice?"
Chiyu Ma, Enpei Zhang, Yilun Zhao, Wenjun Liu, Yaning Jia, Peijun Qing, Lin Shi, Arman Cohan, Yujun Yan, Soroush Vosoughi.
EMNLP 2025 |
language modelings evaluation reasoning
"MetaFaith: Faithful Natural Language Uncertainty Expression in LLMs"
Gabrielle Kaili-May Liu, Gal Yona, Avi Caciularu, Idan Szpektor, Tim G. J. Rudner, Arman Cohan.
EMNLP 2025 | Code |
evaluation safety and alignment llm agents
"SciRIFF: A Resource to Enhance Language Model Instruction-Following over Scientific Literature"
David Wadden, Kejian Shi, Jacob Daniel Morrison, Aakanksha Naik, Shruti Singh, Nitzan Barzilay, Kyle Lo, Tom Hope, Luca Soldaini, Shannon Zejiang Shen, Doug Downey, Hanna Hajishirzi, Arman Cohan
EMNLP 2025 | Code |
post-training evaluation safety and alignment
"Table-R1: Inference-Time Scaling for Table Reasoning"
Zheyuan Yang, Lyuhao Chen, Arman Cohan, Yilun Zhao
EMNLP 2025 | Code |
AI for science post-training
"MCTS-RAG: Enhancing Retrieval-Augmented Generation with Monte Carlo Tree Search"
Yunhai Hu, Yilun Zhao, Chen Zhao, Arman Cohan
EMNLP 2025, Findings | Code |
reasoning post-training efficiency and scalability of LLMs
"MSRS: Evaluating Multi-Source Retrieval-Augmented Generation"
Rohan Phanse, Yijie Zhou, Kejian Shi, Wencai Zhang, Yixin Liu, Yilun Zhao, Arman Cohan.
COLM 2025 | Code |
evaluation reasoning
"One-shot Optimized Steering Vectors Mediate Safety-relevant Behaviors in LLMs"
Jacob Dunefsky, Arman Cohan.
COLM 2025 |
evaluation reasoning language modelings
"Understanding Reference Policies in Direct Preference Optimization"
Yixin Liu, Pengfei Liu, Arman Cohan.
NAACL 2025, findings | Code |
reinforcement learning preference learning document understanding
"TOMATO: Assessing Visual Temporal Reasoning Capabilities in Multimodal Foundation Models"
Ziyao Shangguan, Chuhan Li, Yuxuan Ding, Yanan Zheng, Yilun Zhao, Tesca Fitzgerald, Arman Cohan.
ICLR 2025 | Code |
multimodal vision evaluation
"ReIFE: Re-evaluating Instruction-Following Evaluation"
Yixin Liu, Kejian Shi, Alexander Fabbri, Yilun Zhao, Peifeng Wang, Chien-Sheng Wu, Shafiq Joty, Arman Cohan.
NAACL 2025 | Code |
evaluation foundation models document understanding
"FollowIR: Evaluating and Teaching Information Retrieval Models to Follow Instructions"
Orion Weller, Benjamin Chang, Sean MacAvaney, Kyle Lo, Arman Cohan, Benjamin Van Durme, Dawn Lawrie, Luca Soldaini.
NAACL 2025 |
evaluation training methods document understanding
"IRIS: Interactive Research Ideation System for Accelerating Scientific Discovery"
Aniketh Garikaparthi, Manasi Patwardhan, Lovekesh Vig, Arman Cohan.
ACL 2025 Demo |
scientific-discovery agent
"MIR: Methodology Inspiration Retrieval for Scientific Research Problems"
Aniketh Garikaparthi, Manasi Patwardhan, Aditya Sanjiv Kanade, Aman Hassan, Lovekesh Vig, Arman Cohan.
ACL 2025 |
retrieval scientific-discovery
"SciVer: Evaluating Foundation Models for Multimodal Scientific Claim Verification"
Chengye Wang, Yifei Shen, Zexi Kuang, Arman Cohan, Yilun Zhao.
ACL 2025 |
benchmark multimodal
"AbGen: Evaluating Large Language Models in Ablation Study Design and Evaluation for Scientific Research"
Yilun Zhao, Weiyuan Chen, Zhijian Xu, Manasi Patwardhan, Chengye Wang, Yixin Liu, Lovekesh Vig, Arman Cohan.
ACL 2025 |
benchmark evaluation
"HumanEval Pro and MBPP Pro: Evaluating Large Language Models on Self-invoking Code Generation Task"
Zhaojian Yu, Yilun Zhao, Arman Cohan, Xiaoping Zhang.
ACL Findings 2025 |
benchmark code-generation evaluation
"Physics: Benchmarking Foundation Models on University-Level Physics Problem Solving"
Kaiyue Feng, Yilun Zhao, Yixin Liu, Tianyu Yang, Chen Zhao, John Sous, Arman Cohan.
ACL Findings 2025 |
benchmark specialized domains
"Can Multimodal Foundation Models Understand Schematic Diagrams? An Empirical Study on Information-Seeking QA over Scientific Papers"
Yilun Zhao, Chengye Wang, Chuhan Li, Arman Cohan.
ACL Findings 2025 |
multimodal evaluation
"Can LLMs Identify Critical Limitations within Scientific Research? A Systematic Evaluation on AI Research Papers"
Zhijian Xu, Yilun Zhao, Manasi Patwardhan, Lovekesh Vig, Arman Cohan.
ACL 2025 |
evaluation
"TESS 2: A Large-Scale Generalist Diffusion Language Model"
Jaesung Tae, Hamish Ivison, Sachin Kumar, Arman Cohan.
ACL 2025 |
language-modeling diffusion
"Ref-Long: Benchmarking the Long-context Referencing Capability of Long-context Language Models"
Junjie Wu, Gefei Gu, Yanan Zheng, Dit-Yan Yeung, Arman Cohan.
ACL 2025 |
benchmark evaluation
"MDCure: A Scalable Pipeline for Multi-Document Instruction-Following"
Gabrielle Kaili-May Liu, Bowen Shi, Avi Caciularu, Idan Szpektor, Arman Cohan.
ACL 2025 |
instruction-following multi-document
"ChemAgent: Self-updating Library in Large Language Models Improves Chemical Reasoning"
Xiangru Tang, Tianyu Hu, Muyang Ye, Yanjun Shao, Xunjian Yin, Siru Ouyang, Wangchunshu Zhou, Pan Lu, Zhuosheng Zhang, Yilun Zhao, Arman Cohan, Mark Gerstein.
ICLR 2025 | Code |
specialized domains document understanding LLM agents
"MMVU: Measuring Expert-Level Multi-Discipline Video Understanding"
Yilun Zhao, Lujing Xie, Haowei Zhang, Guo Gan, Yitao Long, Zhiyuan Hu, Tongyan Hu, Weiyuan Chen, Chuhan Li, Junyang Song, Zhijian Xu, Chengye Wang, Weifeng Pan, Ziyao Shangguan, Xiangru Tang, Zhenwen Liang, Yixin Liu, Chen Zhao, Arman Cohan.
Preprint | Code |
foundation models evaluation specialized domains
"RouterRetriever: Exploring the Benefits of Routing over Multiple Expert Embedding Models"
Hyunji Lee, Luca Soldaini, Arman Cohan, Minjoon Seo, Kyle Lo.
AAAI 2025 |
document understanding evaluation specialized domains

2024

"FOLIO: Natural Language Reasoning with First-Order Logic"
Simeng Han, Hailey Schoelkopf, Yilun Zhao, Zhenting Qi, Martin Riddell, Luke Benson, Lucy Sun, E. Zubova, Yujie Qiao, Matthew Burtell, David Peng, Jonathan Fan, Yixin Liu, Brian Wong, Malcolm Sailor, Ansong Ni, Linyong Nan, Jungo Kasai, Tao Yu, Rui Zhang, Shafiq R. Joty, Alexander R. Fabbri, Wojciech Kryscinski, Xi Victoria Lin, Caiming Xiong, Dragomir R. Radev.
EMNLP 2024 |
- evaluation - training methods - interpretability
"Bayesian Calibration of Win Rate Estimation with LLM Evaluators"
Yicheng Gao, Gonghan Xu, Zhe Wang, Arman Cohan.
EMNLP 2024 |
evaluation foundation models training methods
"FinDVer: Explainable Claim Verification over Long and Hybrid-content Financial Documents"
Yilun Zhao, Yitao Long, Tintin Jiang, Chengye Wang, Weiyuan Chen, Hongjun Liu, Xiangru Tang, Yiming Zhang, Chen Zhao, Arman Cohan.
EMNLP 2024 |
document understanding evaluation specialized domains
"SciDQA: A Deep Reading Comprehension Dataset over Scientific Papers"
Shruti Singh, Nandan Sarkar, Arman Cohan.
EMNLP 2024 |
document understanding evaluation foundation models
"P-FOLIO: Evaluating and Improving Logical Reasoning with Abundant Human-Written Reasoning Chains"
Simeng Han, Aaron Yu, Rui Shen, Zhenting Qi, Martin Riddell, Wenfei Zhou, Yujie Qiao, Yilun Zhao, Semih Yavuz, Ye Liu, Shafiq Joty, Yingbo Zhou, Caiming Xiong, Dragomir R. Radev, Rex Ying, Arman Cohan.
EMNLP 2024 Findings |
evaluation training methods interpretability
"Calibrating Long-form Generations from Large Language Models"
Yukun Huang, Yixin Liu, Raghuveer Thirukovalluru, Arman Cohan, Bhuwan Dhingra.
EMNLP 2024 Findings |
evaluation training methods foundation models
"M3SciQA: A Multi-Modal Multi-Document Scientific QA Benchmark for Evaluating Foundation Models"
Chuhan Li, Ziyao Shangguan, Yilun Zhao, Deyuan Li, Yixin Liu, Arman Cohan.
EMNLP 2024 Findings |
foundation models evaluation document understanding
"OMG-QA: Building Open-Domain Multi-Modal Generative Question Answering Systems"
Linyong Nan, Weining Fang, Aylin Rasteh, Pouya Lahabi, Weijin Zou, Yilun Zhao, Arman Cohan.
EMNLP 2024 Industry Track |
- evaluation - document understanding - specialized domains
"MIMIR: A Customizable Agent Tuning Platform for Enhanced Scientific Applications"
Xiangru Tang, Chunyuan Deng, Hanminwang Hanminwang, Haoran Wang, Yilun Zhao, Wenqi Shi, Yi Fung, Wangchunshu Zhou, Jiannan Cao, Heng Ji, Arman Cohan, Mark Gerstein.
EMNLP 2024 Demo |
- LLM agents - training methods - specialized domains
"OpenT2T: An Open-Source Toolkit for Table-to-Text Generation"
Haowei Zhang, Shengyun Si, Yilun Zhao, Lujing Xie, Zhijian Xu, Lyuhao Chen, Linyong Nan, Pengcheng Wang, Xiangru Tang, Arman Cohan.
EMNLP 2024 Demo |
document understanding evaluation foundation models
"TAIL: A Toolkit for Automatic and Realistic Long-Context Large Language Model Evaluation"
Gefei Gu, Yilun Zhao, Ruoxi Ning, Yanan Zheng, Arman Cohan.
EMNLP 2024 Demo |
evaluation foundation models specialized domains
"Observable Propagation: Uncovering Feature Vectors in Transformers"
Jacob Dunefsky, Arman Cohan
ICML 2024 |
interpretability
"NExT: Teaching Large Language Models to Reason about Code Execution"
Ansong Ni, Miltiadis Allamanis, Arman Cohan, Yinlin Deng, Kensen Shi, Charles Sutton, Pengcheng Yin
ICML 2024 |
training methods code generation interpretability
"On Learning to Summarize with Large Language Models as References"
Yixin Liu, Alexander R. Fabbri, Pengfei Liu, Dragomir R. Radev, Arman Cohan.
NAACL 2024 |
foundation models
"Investigating Data Contamination in Modern Benchmarks for Large Language Models"
Chunyuan Deng, Yilun Zhao, Xiangru Tang, Mark B. Gerstein, Arman Cohan.
NAACL 2024 |
evaluation foundation models interpretability
"Struc-Bench: Are Large Language Models Really Good at Generating Complex Structured Data?"
Xiangru Tang, Yiming Zong, Jason Phang, Yilun Zhao, Wangchunshu Zhou, Arman Cohan, Mark Gerstein.
NAACL 2024 |
foundation models evaluation training methods
"Struc-Bench: Are Large Language Models Good at Generating Complex Structured Tabular Data?"
Xiangru Tang, Yiming Zong, Jason Phang, Yilun Zhao, Wangchunshu Zhou, Arman Cohan, Mark Gerstein.
NAACL 2024 |
- evaluation - training methods - document understanding
"On Evaluating the Integration of Reasoning and Action in LLM Agents with Database Question Answering"
Linyong Nan, Ellen Zhang, Weijin Zou, Yilun Zhao, Wenfei Zhou, Arman Cohan.
NAACL 2024 Findings |
LLM agents evaluation document understanding
"Benchmarking Generation and Evaluation Capabilities of Large Language Models for Instruction Controllable Summarization"
Yixin Liu, A. R. Fabbri, Jiawen Chen, Yilun Zhao, Simeng Han, Shafiq R. Joty, Pengfei Liu, Dragomir R. Radev, Chien-Sheng Wu, Arman Cohan.
NAACL 2024 Findings |
evaluation training methods document understanding
"Quantifying Contamination in Evaluating Code Generation Capabilities of Language Models"
Martin Riddell, Ansong Ni, Arman Cohan.
ACL 2024 |
evaluation code generation
"OLMo: Accelerating the Science of Language Models"
Dirk Groeneveld, Iz Beltagy, Pete Walsh, Akshita Bhagia, Rodney Kinney, Oyvind Tafjord, A. Jha, Hamish Ivison, Ian Magnusson, Yizhong Wang, Shane Arora, David Atkinson, Russell Authur, Khyathi Raghavi Chandu, Arman Cohan, Jennifer Dumas, Yanai Elazar, Yuling Gu, Jack Hessel, Tushar Khot, William Merrill, Jacob Daniel Morrison, Niklas Muennighoff, Aakanksha Naik, Crystal Nam, Matthew E. Peters, Valentina Pyatkin, Abhilasha Ravichander, Dustin Schwenk, Saurabh Shah, Will Smith, Emma Strubell, Nishant Subramani, Mitchell Wortsman, Pradeep Dasigi, Nathan Lambert, Kyle Richardson, Luke Zettlemoyer, Jesse Dodge, Kyle Lo, Luca Soldaini, Noah A. Smith, Hanna Hajishirzi.
ACL 2024 |
foundation models
"TaPERA: Enhancing Faithfulness and Interpretability in Long-Form Table QA by Content Planning and Execution-based Reasoning"
Yilun Zhao, Lyuhau Chen, Arman Cohan, Chen Zhao
ACL 2024 |
reasoning evaluation document understanding
FinanceMath: A Knowledge-Intensive Math Reasoning Dataset in Finance Domains
Yilun Zhao, Hongjun Liu, Yitao Long, Rui Zhang, Chen Zhao, Arman Cohan
ACL 2024 |
dataset reasoning evaluation document understanding specialized domains
"DocMath-Eval: Evaluating Math Reasoning Capabilities of LLMs in Understanding Financial Documents"
Yilun Zhao, Yitao Long, Hongjun Liu, Ryo Kamoi, Linyong Nan, Lyuhao Chen, Yixin Liu, Xiangru Tang, Rui Zhang, Arman Cohan.
ACL 2024 |
- evaluation - document understanding - specialized domains
"Rethinking Efficient Multilingual Text Summarization Meta-Evaluation"
Rilyn Han, Jiawen Chen, Yixin Liu, Arman Cohan.
ACL 2024 |
- evaluation - document understanding - specialized domains
"Unveiling the Spectrum of Data Contamination in Language Models: A Survey from Detection to Remediation"
Chunyuan Deng, Yilun Zhao, Yuzhao Heng, Yitong Li, Jiannan Cao, Xiangru Tang, Arman Cohan.
ACL 2024 |
evaluation training methods foundation models
"Unveiling the Spectrum of Data Contamination in Language Models: A Survey from Detection to Remediation"
Chunyuan Deng, Yilun Zhao, Yuzhao Heng, Yitong Li, Jiannan Cao, Xiangru Tang, Arman Cohan
ACL 2024 findings |
evaluation contamination survey
"MedAgents: Large Language Models as Collaborators for Zero-shot Medical Reasoning"
Xiangru Tang, Anni Zou, Zhuosheng Zhang, Ziming Li, Yilun Zhao, Xingyao Zhang, Arman Cohan, Mark Gerstein
ACL 2024 findings |
LM agents specialized domains reasoning
"On the Benefits of Fine-Grained Loss Truncation: A Case Study on Factuality in Summarization"
Lorenzo Jaime Yu Flores, Arman Cohan.
EACL 2024 |
evaluation training methods document understanding
"Step-Back Profiling: Distilling User History for Personalized Scientific Writing"
Xiangru Tang, Xingyao Zhang, Yanjun Shao, Jie Wu, Yilun Zhao, Arman Cohan, Ming Gong, Dongmei Zhang, Mark Gerstein
AI4Science workshop @IJCAI 2024 |
personalization writing generation
"Evaluating LLMs at Detecting Errors in LLM Responses"
Ryo Kamoi, Sarkar Snigdha Sarathi Das, Renze Lou, Jihyun Janice Ahn, Yilun Zhao, Xiaoxin Lu, Nan Zhang, Yusen Zhang, Ranran Haoran Zhang, Sujeeth Reddy Vummanthala, Salika Dave, Shaobo Qin, Arman Cohan, Wenpeng Yin, Rui Zhang.
COLM 2024 |
evaluation foundation models interpretability
"L2CEval: Evaluating Language-to-Code Generation Capabilities of Large Language Models"
Ansong Ni, Pengcheng Yin, Yilun Zhao, Martin Riddell, Troy Feng, Rui Shen, Stephen Yin, Ye Liu, Semih Yavuz, Caiming Xiong, Shafiq R. Joty, Yingbo Zhou, Dragomir R. Radev, Arman Cohan.
TACL |
evaluation code generation
"Prioritizing Safeguarding Over Autonomy: Risks of LLM Agents for Science"
Xiangru Tang, Qiao Jin, Kunlun Zhu, Tongxin Yuan, Yichi Zhang, Wangchunshu Zhou, Meng Qu, Yilun Zhao, Jian Tang, Zhuosheng Zhang, Arman Cohan, Zhiyong Lu, Mark B. Gerstein.
Preprint |
LLM agents specialized domains evaluation
"SciRIFF: A Resource to Enhance Language Model Instruction-Following over Scientific Literature"
David Wadden, Kejian Shi, Jacob Daniel Morrison, Aakanksha Naik, Shruti Singh, Nitzan Barzilay, Kyle Lo, Tom Hope, Luca Soldaini, Shannon Zejiang Shen, Doug Downey, Hanna Hajishirzi, Arman Cohan.
Preprint |
- foundation models - evaluation - document understanding
"MetaMath: Integrating Natural Language and Code for Enhanced Mathematical Reasoning in Large Language Models"
Xuyuan Xiong, Simeng Han, Ziyue Zhou, Arman Cohan.
Preprint |
code generation document understanding LLM agents

2023

"A Question Answering Framework for Decontextualizing User-facing Snippets from Scientific Documents"
Benjamin Newman, Luca Soldaini, Raymond Fok, Arman Cohan, Kyle Lo.
EMNLP 2023 |
document understanding evaluation LLM agents
"QTSumm: A New Benchmark for Query-Focused Table Summarization"
Yilun Zhao, Zhenting Qi, Linyong Nan, Boyu Mi, Yixin Liu, Weijin Zou, Simeng Han, Xiangru Tang, Yumo Xu, Arman Cohan, Dragomir R. Radev.
EMNLP 2023 |
document understanding evaluation foundation models
"Enhancing Text-to-SQL Capabilities of Large Language Models: A Study on Prompt Design Strategies"
Linyong Nan, Yilun Zhao, Weijin Zou, Narutatsu Ri, Jaesung Tae, Ellen Zhang, Arman Cohan, Dragomir R. Radev.
EMNLP 2023 Findings |
- foundation models - training methods - document understanding
"Medical Text Simplification: Optimizing for Readability with Unlikelihood Training and Reranked Beam Search Decoding"
Lorenzo Jaime Yu Flores, Heyuan Huang, Kejian Shi, Sophie Chheang, Arman Cohan.
EMNLP 2023 Findings |
specialized domains document understanding evaluation
"Enhancing Few-shot Text-to-SQL Capabilities of Large Language Models: A Study on Prompt Design Strategies"
Linyong Nan, Yilun Zhao, Weijin Zou, Narutatsu Ri, Jaesung Tae, E. Zhang, Arman Cohan, Dragomir R. Radev.
EMNLP 2023 Findings |
- foundation models - evaluation - document understanding
"Open Domain Multi-document Summarization: A Comprehensive Study of Model Brittleness under Retrieval"
John Giorgi, Luca Soldaini, Bo Wang, Gary Bader, Kyle Lo, Lucy Lu Wang, Arman Cohan.
EMNLP 2023 Findings |
document understanding evaluation training methods
"Investigating Table-to-Text Generation Capabilities of LLMs in Real-World Information Seeking Scenarios"
Yilun Zhao, Haowei Zhang, Shengyun Si, Linyong Nan, Xiangru Tang, Arman Cohan.
EMNLP 2023 Industry Track |
document understanding evaluation foundation models
"Investigating Table-to-Text Generation Capabilities of Large Language Models in Real-World Information Seeking Scenarios"
Yilun Zhao, Haowei Zhang, Shengyun Si, Linyong Nan, Xiangru Tang, Arman Cohan.
EMNLP 2023 industry track |
document understanding LLM agents foundation models
"Peek Across: Improving Multi-Document Modeling via Cross-Document Question-Answering"
Avi Caciularu, Matthew E. Peters, J. Goldberger, Ido Dagan, Arman Cohan.
ACL 2023 |
multi-document modeling evaluation training methods
"OpenRT: An Open-source Framework for Reasoning Over Tabular Data"
Yilun Zhao, Boyu Mi, Zhenting Qi, Linyong Nan, Minghao Guo, Arman Cohan, Dragomir R. Radev.
ACL 2023 Demo |
document understanding evaluation specialized domains
"TESS: Text-to-Text Self-Conditioned Simplex Diffusion"
Rabeeh Karimi Mahabadi, Jaesung Tae, Hamish Ivison, J. Henderson, Iz Beltagy, Matthew E. Peters, Arman Cohan.
EACL 2023 |
training methods document understanding foundation models
"LongEval: Guidelines for Human Evaluation of Faithfulness in Long-form Summarization"
Kalpesh Krishna, Erin Bransom, Bailey Kuehl, Mohit Iyyer, Pradeep Dasigi, Arman Cohan, Kyle Lo.
EACL 2023 |
evaluation document understanding specialized domains
"Aligning Factual Consistency for Clinical Studies Summarization through Reinforcement Learning"
Xiangru Tang, Arman Cohan, M. Gerstein.
Clinical Natural Language Processing Workshop 2023 |
- document understanding - evaluation - training methods
"Natural language processing in radiology: Clinical applications and future directions."
Pratheek S Bobba, A. Sailer, J. Pruneski, Spencer Beck, Ali Mozayan, Sara Mozayan, Jennifer Arango, Arman Cohan, S. Chheang.
Clinical imaging 2023 |
specialized domains document understanding evaluation
"When do Generative Query and Document Expansions Fail? A Comprehensive Study Across Methods, Retrievers, and Datasets"
Orion Weller, Kyle Lo, David Wadden, Dawn J Lawrie, Benjamin Van Durme, Arman Cohan, Luca Soldaini.
Findings 2023 |
evaluation training methods document understanding
"ReFIT: Relevance Feedback from a Reranker during Inference"
R. Reddy, Pradeep Dasigi, Md Arafat Sultan, Arman Cohan, Avirup Sil, Heng Ji, Hannaneh Hajishirzi.
Preprint |
document understanding evaluation training methods
"ODSum: New Benchmarks for Open Domain Multi-Document Summarization"
Yijie Zhou, Kejian Shi, Wencai Zhang, Yixin Liu, Yilun Zhao, Arman Cohan.
Preprint |
document understanding evaluation training methods
"The Semantic Scholar Open Data Platform"
Rodney Michael Kinney, Chloe Anastasiades, Russell Authur, Iz Beltagy, Jonathan Bragg, Alexandra Buraczynski, Isabel Cachola, Stefan Candra, Yoganand Chandrasekhar, Arman Cohan, Miles Crawford, Doug Downey, Jason Dunkelberger, Oren Etzioni, Rob Evans, Sergey Feldman, Joseph Gorney, D. Graham, F.Q. Hu, Regan Huff, Daniel King, Sebastian Kohlmeier, Bailey Kuehl, Michael Langan, Daniel Lin, Haokun Liu, Kyle Lo, Jaron Lochner, Kelsey MacMillan, Tyler Murray, Christopher Newell, Smita R Rao, Shaurya Rohatgi, P. Sayre, Zejiang Shen, Amanpreet Singh, Luca Soldaini, Shivashankar Subramanian, A. Tanaka, Alex D Wade, Linda M. Wagner, Lucy Lu Wang, Christopher Wilhelm, Caroline Wu, Jiangjiang Yang, Angele Zamarron, Madeleine van Zuylen, Daniel S. Weld.
Preprint |
document understanding foundation models evaluation
"ML-Bench: Evaluating Large Language Models for Code Generation in Repository-Level Machine Learning Tasks"
Yuliang Liu, Xiangru Tang, Zefan Cai, Junjie Lu, Yichi Zhang, Yan Shao, Zexuan Deng, Helan Hu, Zengxian Yang, Kaikai An, Ruijun Huang, Shuzheng Si, Sheng Chen, Haozhe Zhao, Zheng Li, Liang Chen, Yiming Zong, Yan Wang, Tianyu Liu, Zhiwei Jiang, Baobao Chang, Yujia Qin, Wangchunshu Zhou, Yilun Zhao, Arman Cohan, Mark B. Gerstein.
Preprint |
code generation evaluation LLM agents

2022

"SciFact-Open: Towards open-domain scientific claim verification"
David Wadden, Kyle Lo, Bailey Kuehl, Arman Cohan, Iz Beltagy, Lucy Lu Wang, Hannaneh Hajishirzi.
EMNLP 2022 |
document understanding evaluation specialized domains
"Open Domain Multi-document Summarization: A Comprehensive Study of Model Brittleness under Retrieval"
John Giorgi, Luca Soldaini, Bo Wang, Gary Bader, Kyle Lo, Lucy Lu Wang, Arman Cohan.
EMNLP 2022 |
- document understanding - evaluation - training methods
"SciRepEval: A Multi-Format Benchmark for Scientific Document Representations"
Amanpreet Singh, Mike D'Arcy, Arman Cohan, Doug Downey, Sergey Feldman.
EMNLP 2022 |
document understanding evaluation training methods
"Generating Scientific Claims for Zero-Shot Scientific Fact Checking"
Dustin Wright, David Wadden, Kyle Lo, Bailey Kuehl, Arman Cohan, Isabelle Augenstein, Lucy Lu Wang.
ACL 2022 |
- document understanding - evaluation - specialized domains
"Improving the Generalizability of Depression Detection by Leveraging Clinical Questionnaires"
Thong Nguyen, Andrew Yates, Ayah Zirikly, Bart Desmet, Arman Cohan.
ACL 2022 |
- interpretability - evaluation - specialized domains
"Zero- and Few-Shot NLP with Pretrained Language Models"
Iz Beltagy, Arman Cohan, Robert Logan IV, Sewon Min, Sameer Singh.
ACL 2022 |
training methods foundation models evaluation
"Revisiting the Gold Standard: Grounding Summarization Evaluation with Robust Human Evaluation"
Yixin Liu, Alexander R. Fabbri, Pengfei Liu, Yilun Zhao, Linyong Nan, Ruilin Han, Simeng Han, Shafiq R. Joty, Chien-Sheng Wu, Caiming Xiong, Dragomir R. Radev.
ACL 2022 |
evaluation foundation models document understanding
"Embedding Recycling for Language Models"
Jon Saad-Falcon, Amanpreet Singh, Luca Soldaini, Mike D'Arcy, Arman Cohan, Doug Downey.
Findings 2022 |
evaluation training methods foundation models
"Overview of the First Shared Task on Multi Perspective Scientific Document Summarization (MuP)"
Arman Cohan, Guy Feigenblat, Tirthankar Ghosal, Michal Shmueli-Scheuer.
SDP 2022 |
document understanding evaluation foundation models
"Overview of the Third Workshop on Scholarly Document Processing"
Arman Cohan, Guy Feigenblat, Dayne Freitag, Tirthankar Ghosal, Drahomira Herrmannova, Petr Knoth, Kyle Lo, Philipp Mayr, Michal Shmueli-Scheuer, Anita de Waard, Lucy Lu Wang.
SDP 2022 |
document understanding evaluation foundation models

2021

"Cross-Document Language Modeling"
Avi Caciularu, Arman Cohan, Iz Beltagy, Matthew E. Peters, Arie Cattan, Ido Dagan.
EMNLP 2021 |
foundation models training methods document understanding
"FLEX: Unifying Evaluation for Few-Shot NLP"
Jonathan Bragg, Arman Cohan, Kyle Lo, Iz Beltagy.
NeurIPS 2021 |
evaluation foundation models training methods
"A Dataset of Information-Seeking Questions and Answers Anchored in Research Papers"
Pradeep Dasigi, Kyle Lo, Iz Beltagy, Arman Cohan, Noah A. Smith, Matt Gardner.
NAACL 2021 |
document understanding evaluation specialized domains
"Beyond Paragraphs: NLP for Long Sequences"
Iz Beltagy, Arman Cohan, Hannaneh Hajishirzi, Sewon Min, Matthew E. Peters.
NAACL 2021 |
document understanding foundation models evaluation
"Long Context Question Answering via Supervised Contrastive Learning"
Avi Caciularu, Ido Dagan, J. Goldberger, Arman Cohan.
NAACL 2021 |
long-context question answering evaluation training methods
"Multi-Vector Models with Textual Guidance for Fine-Grained Scientific Document Similarity"
Sheshera Mysore, Arman Cohan, Tom Hope.
NAACL 2021 |
document understanding evaluation training methods
"MultiCite: Modeling realistic citations requires moving beyond the single-sentence single-label setting"
Anne Lauscher, B. Ko, Bailey Kuehl, Sophie Johnson, David Jurgens, Arman Cohan, Kyle Lo.
NAACL 2021 |
document understanding evaluation specialized domains
"PRIMERA: Pyramid-based Masked Sentence Pre-training for Multi-document Summarization"
Wen Xiao, Iz Beltagy, G. Carenini, Arman Cohan.
ACL 2021 |
foundation models evaluation document understanding
"Simplified Data Wrangling with ir_datasets"
Sean MacAvaney, Andrew Yates, Sergey Feldman, Doug Downey, Arman Cohan, Nazli Goharian.
Annual International ACM SIGIR Conference on Research and Development in Information Retrieval 2021 |
document understanding specialized domains evaluation
"Two Contrasting Data Annotation Paradigms for Subjective NLP Tasks"
David Wadden, Kyle Lo, Lucy Lu Wang, Arman Cohan, Iz Beltagy, Hannaneh Hajishirzi.
NAACL-HLT 2021 |
document understanding evaluation specialized domains
"LongChecker: Improving scientific claim verification by modeling full-abstract context"
David Wadden, Kyle Lo, Lucy Lu Wang, Arman Cohan, Iz Beltagy, Hannaneh Hajishirzi.
Preprint |
document understanding evaluation specialized domains
"Overview of the Second Workshop on Scholarly Document Processing"
Iz Beltagy, Arman Cohan, Guy Feigenblat, Dayne Freitag, Tirthankar Ghosal, Keith B. Hall, Drahomira Herrmannova, Petr Knoth, Kyle Lo, Philipp Mayr, Robert M. Patton, Michal Shmueli-Scheuer, A. Waard, Kuansan Wang, Lucy Lu Wang.
Preprint |
document understanding foundation models evaluation
"PRIMER: Pyramid-based Masked Sentence Pre-training for Multi-document Summarization"
Wen Xiao, Iz Beltagy, G. Carenini, Arman Cohan.
Preprint |
foundation models training methods document understanding
"Utilizing Evidence Spans via Sequence-Level Contrastive Learning for Long-Context Question Answering"
Avi Caciularu, Ido Dagan, J. Goldberger, Arman Cohan.
Preprint |
- training methods - evaluation - document understanding

2020

"Fact or Fiction: Verifying Scientific Claims"
David Wadden, Kyle Lo, Lucy Lu Wang, Shanchuan Lin, Madeleine van Zuylen, Arman Cohan, Hannaneh Hajishirzi.
EMNLP 2020 |
document understanding evaluation specialized domains
"SLEDGE: A Simple Yet Effective Zero-Shot Baseline for Coronavirus Scientific Knowledge Search"
Sean MacAvaney, Arman Cohan, Nazli Goharian.
EMNLP 2020 |
document understanding evaluation specialized domains
"SPECTER: Document-level Representation Learning using Citation-informed Transformers"
Arman Cohan, Sergey Feldman, Iz Beltagy, Doug Downey, Daniel S. Weld.
ACL 2020 |
document understanding evaluation foundation models
"Ranking Significant Discrepancies in Clinical Reports"
Sean MacAvaney, Arman Cohan, Nazli Goharian, Ross W. Filice.
European Conference on Information Retrieval 2020 |
- document understanding - specialized domains - evaluation
"TLDR: Extreme Summarization of Scientific Documents"
Isabel Cachola, Kyle Lo, Arman Cohan, Daniel S. Weld.
Findings 2020 |
document understanding evaluation training methods
"GUIR @ LongSumm 2020: Learning to Generate Long Summaries from Scientific Documents"
Sajad Sotudeh Gharebagh, Arman Cohan, Nazli Goharian.
SDP 2020 |
document understanding evaluation training methods
"On Generating Extended Summaries of Long Documents"
Sajad Sotudeh, Arman Cohan, Nazli Goharian.
SDU@AAAI 2020 |
document understanding evaluation training methods
"ParsiNLU: A Suite of Language Understanding Challenges for Persian"
Daniel Khashabi, Arman Cohan, Siamak Shakeri, Pedram Hosseini, Pouya Pezeshkpour, Malihe Alikhani, Moin Aminnaseri, Marzieh Bitaab, Faeze Brahman, Sarik Ghazarian, Mozhdeh Gheini, Arman Kabiri, Rabeeh Karimi Mahabadi, Omid Memarrast, Ahmadreza Mosallanezhad, Erfan Noury, Shahab Raji, Mohammad Sadegh Rasooli, Sepideh Sadeghi, Erfan Sadeqi Azer, Niloofar Safi Samghabadi, Mahsa Shafaei, Saber Sheybani, Ali Tazarv, Yadollah Yaghoobzadeh.
Transactions of the Association for Computational Linguistics 2020 |
document understanding evaluation specialized domains
"Longformer: The Long-Document Transformer"
Iz Beltagy, Matthew E. Peters, Arman Cohan.
Preprint |
foundation models evaluation document understanding
"SLEDGE: A Simple Yet Effective Baseline for Coronavirus Scientific Knowledge Search"
Sean MacAvaney, Arman Cohan, Nazli Goharian.
Preprint |
document understanding evaluation specialized domains

2019

"Pretrained Language Models for Sequential Sentence Classification"
Arman Cohan, Iz Beltagy, Daniel King, Bhavana Dalvi, Daniel S. Weld.
EMNLP 2019 |
training methods foundation models
"SciBERT: A Pretrained Language Model for Scientific Text"
Iz Beltagy, Kyle Lo, Arman Cohan.
EMNLP 2019 |
foundation models language modeling scientific text
"Structural Scaffolds for Citation Intent Classification in Scientific Publications"
Arman Cohan, Bridger Waleed Ammar, Madeleine van Zuylen, Field Cady.
NAACL 2019 |
training methods scientific text specialized domains
"SUPP.AI: finding evidence for supplement-drug interactions"
Lucy Lu Wang, Oyvind Tafjord, Arman Cohan, Sarthak Jain, Sam Skjonsberg, Carissa Schoenick, Nick Botner, Bridger Waleed Ammar.
ACL 2019 |
specialized domains information extraction
"Extracting evidence of supplement-drug interactions from literature"
Lucy Lu Wang, Oyvind Tafjord, Sarthak Jain, Arman Cohan, Sam Skjonsberg, Carissa Schoenick, Nick Botner, Bridger Waleed Ammar.
BioNLP workshop |
specialized domains information extraction
"CEDR: Contextualized Embeddings for Document Ranking"
Sean MacAvaney, Andrew Yates, Arman Cohan, Nazli Goharian.
SIGIR 2019 |
retrieval representation learning
"Ontology-Aware Clinical Abstractive Summarization"
Sean MacAvaney, Sajad Sotudeh, Arman Cohan, Nazli Goharian, Ish A. Talati, Ross W. Filice.
SIGIR 2019 |
scientific text summarization specialized domains
"Text Summarization and Categorization for Scientific and Health-Related Data"
Arman Cohan.
SIGIR 2019 |
specialized domains summarization
"Contextualized Word Representations for Document Re-Ranking"
Sean MacAvaney, Andrew Yates, Arman Cohan, Nazli Goharian.
Preprint |
retrieval representation learning
"SciBERT: Pretrained Contextualized Embeddings for Scientific Text"
Iz Beltagy, Arman Cohan, Kyle Lo.
Preprint |
foundation models evaluation specialized domains

Note: This publication list is partially automatically derived from Semantic Scholar. While we make efforts to manually verify and correct entries, some errors may still be present.

Filter by category:

Filter by publication type:

Search: