An interdisciplinary, randomized, single-blind evaluation of state-of-the-art large language models for their implications and risks in medical diagnosis and management

Peikai Chen, Jifu Cai, Jiaying Zhou, Shaoxi Chen, Chenguang Xu, Lihua Yuan, Xiaoying Dai, Xiaowei Chen, Yumei Wei, Xia Li, Shaofeng Gong, Xiaolong Liang, Jiancheng Yang, Jun‐O Jin, Kai Dai, Yuzhen Cui, Guan‐Ming Kuang, Ju-Jun Xie, Libing Luo, Haibing Xiao, Shijie Yin, H. J. Yang, Yulan Yan, Jianliang Chen, Yi‐Hua Chen, Q.j. Zhang, Qingshan Zhou, Lina Zhao, Min Wu, Xin Tang, Rong Lei, Zanxin Wang, Wenbin Qiu, Yanli Wang, Liwen Cui, Yong Hu, Huiren Tao, Nan Wu, Pearl Pai, M. Wei, Michael To, Kmc CheungmedRxiv 2025 · DOI

Abstract

Background: State-of-the-art (SOTA) large language models (LLMs) are poised to revolutionize clinical medicine by transforming diagnostic, therapeutic, and interdisciplinary reasoning. Despite their promising capabilities, rigorous benchmarking of these models is essential to address concerns about their clinical proficiency and safety, particularly in high-risk environments.

Methods: This study implemented a multi-disciplinary, randomized, single-blind evaluation framework involving 27 experienced specialty clinicians with an average of 25.9 years of practice. The assessment covered 685 simulated and real clinical cases across 13 subspecialties, including both common and rare conditions. Evaluators rated LLM responses on medical strength (0–10 scale, where > 9.5 signified leading expert proficiency) and hallucination severity (0–5 scale for fabricated or misleading medical elements). Seven SOTA LLMs were tested, including top-ranked models from the ARENA leaderboard, with statistical analyses applied to adjust for confounders such as response length.

Findings: The evaluation revealed clinical plausibility in general-purpose LLMs, with Gemini 2.0 Flash leading raw scores and DeepSeek R1 excelling in adjusted analyses. Top models demonstrated proficiency comparable to a physician of 6 years post qualification experience (score ∼6.0), yet significant risks were noted. Instances of incompetence (scores ≤4) were detected across specialties, and 40 hallucination instances involving fabricated conditions, medications, and classification errors. These findings underscore the importance of implementing stringent safeguards to mitigate potential adverse outcomes in clinical applications.

Interpretation: While SOTA LLMs show substantial promise in enhancing clinical reasoning and decision-making, their unguarded application in medicine could present serious risks, such as misinformation and diagnostic errors. Human expert oversight remains crucial, particularly given reported incompetence and hallucination risks. Larger, multi-center studies are warranted to evaluate their real-world performance and track their evolution before broader clinical adoption.

Introduction

Most existing AI medical benchmarks, such as MedQA or MMLU, rely on multiple-choice formats. These are prone to "overfitting" and fail to capture the nuances of clinical reasoning required in a hospital setting. Furthermore, many studies do not account for "response length bias"—the tendency for human evaluators to give higher scores to longer, more authoritative-sounding answers, regardless of accuracy. This study aims to bridge this gap by using open-ended clinical scenarios and a single-blind methodology where expert clinicians rated model responses without knowing which AI produced them.

Methods

The researchers implemented a three-round evaluation framework involving 27 veteran clinicians. The study utilized a custom dataset of 685 clinical cases, split between OSCE (structured board-exam style) and real-world cases drawn from hospital records across 13 subspecialties.

The multi-round evaluation framework for assessing LLMs in clinical scenarios, including question design, blind rating by experts, and statistical adjustment for response length.

Seven models—including Gemini 2.0 Flash, ChatGPT-4o, and DeepSeek-R1 (671B)—generated responses. Clinicians rated these on two scales: Medical Strength (0–10) and Hallucination Severity (0–5). Because Gemini 2.0 Flash produced responses significantly longer than its peers, the team used multiple linear regression and mixed-effects modeling to adjust scores for response length, ensuring a fair comparison.

Results

The evaluation demonstrated that top-tier LLMs are approaching the proficiency of mid-career physicians. The top models achieved a mean score of ~6.0, equivalent to a physician with 3–6 years of post-qualification experience.

Performance of SOTA LLMs across 13 subspecialties for both real-world cases (A) and OSCE questions (B). Raw strength scores (0-10) are shown for each model.

Key Findings

  • The "Length Confounder": Longer AI responses tend to receive higher raw scores. When unadjusted, Gemini 2.0 Flash led the pack. However, after adjusting for length, DeepSeek R1 emerged as the most robust model for clinical reasoning efficiency.
  • The Incompetence Floor: Despite high average scores, all models showed "instances of incompetence" (scores ≤ 4) in approximately 7.2% of cases. Gemini 2.0 had the lowest rate of incompetence (2.7%).
  • Dangerous Hallucinations: Researchers documented 40 specific instances of hallucinations, including fabricated drug names and incorrect dosages (e.g., recommending unsafe pediatric doses).
  • Rare Diseases: LLMs showed high value in rare disease diagnosis, often synthesizing vast amounts of literature more effectively than generalist physicians.

Discussion and Conclusion

While LLMs are powerful reasoning engines, they are not infallible knowledge databases. The study highlights a persistent risk of clinical incompetence and hallucinations that could prove dangerous without human oversight. The authors conclude that while LLMs show substantial promise in enhancing clinical reasoning, they must be used with stringent human-in-the-loop safeguards. Future work should involve larger, multi-center studies to track the evolution of these models before broader clinical adoption.

Summary updated

Abstract

Abstract Background State-of-the-art (SOTA) large language models (LLMs) are poised to revolutionize clinical medicine by transforming diagnostic, therapeutic, and interdisciplinary reasoning. Despite their promising capabilities, rigorous benchmarking of these models is essential to address concerns about their clinical proficiency and safety, particularly in high-risk environments. Methods This study implemented a multi-disciplinary, randomized, single-blind evaluation framework involving 27 experienced specialty clinicians with an average of 25.9 years of practice. The assessment covered 685 simulated and real clinical cases across 13 subspecialties, including both common and rare conditions. Evaluators rated LLM responses on medical strength (0–10 scale, where > 9.5 signified leading expert proficiency) and hallucination severity (0–5 scale for fabricated or misleading medical elements). Seven SOTA LLMs were tested, including top-ranked models from the ARENA leaderboard, with statistical analyses applied to adjust for confounders such as response length. Findings The evaluation revealed clinical plausibility in general-purpose LLMs, with Gemini 2.0 Flash leading raw scores and DeepSeek R1 excelling in adjusted analyses. Top models demonstrated proficiency comparable to a physician of 6 years post qualification experience (score ∼6.0), yet significant risks were noted. Instances of incompetence (scores ≤4) were detected across specialties, and 40 hallucination instances involving fabricated conditions, medications, and classification errors. These findings underscore the importance of implementing stringent safeguards to mitigate potential adverse outcomes in clinical applications. Interpretation While SOTA LLMs show substantial promise in enhancing clinical reasoning and decision-making, their unguarded application in medicine could present serious risks, such as misinformation and diagnostic errors. Human expert oversight remains crucial, particularly given reported incompetence and hallucination risks. Larger, multi-center studies are warranted to evaluate their real-world performance and track their evolution before broader clinical adoption.
BibTeX
@misc{chen2025interdisciplinary,
  author = {Peikai Chen and Jifu Cai and Jiaying Zhou and Shaoxi Chen and Chenguang Xu and Lihua Yuan and Xiaoying Dai and Xiaowei Chen and Yumei Wei and Xia Li and Shaofeng Gong and Xiaolong Liang and Jiancheng Yang and Jun‐O Jin and Kai Dai and Yuzhen Cui and Guan‐Ming Kuang and Ju-Jun Xie and Libing Luo and Haibing Xiao and Shijie Yin and H. J. Yang and Yulan Yan and Jianliang Chen and Yi‐Hua Chen and Q.J. Zhang and Qingshan Zhou and Lina Zhao and Min Wu and Xin Tang and Rong Lei and Zanxin Wang and Wenbin Qiu and Yanli Wang and Liwen Cui and Xiangyang Li and Yong Hu and Huiren Tao and Nan Wu and Pearl Pai and M. Wei and Michael To and Kmc Cheung},
  title = {{An interdisciplinary, randomized, single-blind evaluation of state-of-the-art large language models for their implications and risks in medical diagnosis and management}},
  year = {2025},
  doi = {10.1101/2025.06.20.25326623},
  howpublished = {medRxiv},
}
Figure