Large Language Models (LLMs) routinely make headlines for passing medical board exams, diagnosing rare conditions, and even outperforming human specialists on paper. For instance, in a study by Jason Holmes et al., GPT-4 surpassed the average score of human medical physicists on a highly specialized radiation oncology exam (Evaluating large language models on a highly-specialized topic, radiation oncology physics).
Yet, when these same models are deployed in actual hospitals, their performance often stumbles. This disconnect points to a fundamental realism gap in how medical AI is evaluated. Standard benchmarks rely heavily on structured, multiple-choice questions (MCQs) and idealized textbook cases. Real-world clinical practice, by contrast, is messy, open-ended, and deeply collaborative. To build medical AI that is truly safe and effective, researchers are shifting from testing test-taking ability to evaluating end-to-end clinical workflows.
Why Multiple-Choice Exams Mislead
To understand why high exam scores do not guarantee clinical competence, we have to look at what multiple-choice questions actually measure. In an MCQ, the correct answer is already present on the page; the model's task is primarily one of option comparison and elimination.
Yuxuan Zhou et al. exposed this limitation using a framework called MultifacetEval. By testing the exact same medical knowledge point across different cognitive facets—such as independently rectifying an error or verifying a medical fact versus simply picking it from a list—the authors showed that a model's high multiple-choice score often masks a profound inability to generate or verify medical facts independently.
When the structure of a standard exam is removed, the illusion of clinical competence can dissolve entirely. Maxime Griot et al. investigated this by creating a fictional medical universe to isolate genuine clinical reasoning from mere pattern recognition (Pattern Recognition or Medical Knowledge? The Problem with Multiple-Choice Questions in Medicine). Their findings suggest that standard benchmarks heavily reward an LLM's world-class ability to recognize text patterns rather than its capacity to apply medical logic to novel scenarios.
Standard Benchmark (Option Comparison)
[Scenario] -> [Option A] [Option B] [Option C] [Option D] -> Model selects best fit via pattern matching
Real-World Clinical Demand (Generative Reasoning)
[Messy Data] -> Model must synthesize history -> Formulate differential -> Verify facts independently
Textbook Cases vs. Messy Patient Realities
In a textbook or a board exam, patient cases are neatly curated. All irrelevant details are stripped away, and the remaining symptoms point cleanly toward a specific diagnosis. Real patients do not present this way.
As Tassallah Abdullahi et al. note, standardized benchmarks prioritize these idealized cases, failing to reflect the multifaceted reality of diagnosing rare and complex diseases where symptoms overlap and conflict (Learning to Make Rare and Complex Diagnoses With Generative AI Assistance: Qualitative Study of Popular Large Language Models). Furthermore, real patient queries are rarely written in precise medical terminology. The K-QA benchmark, developed by Itay Manes et al., demonstrates that when models are confronted with the informal jargon and scattered symptom descriptions typical of real-world patient queries, even state-of-the-art models struggle to consistently extract and address all essential clinical details.
Crucially, real-world interactions require a model to handle human error and misconceptions. When patients ask questions, they often embed incorrect assumptions. The Cancer-Myth benchmark by Wang Bill Zhu et al. revealed that frontier LLMs fail to correct false presuppositions in patient questions more than 43% of the time, frequently validating or reinforcing harmful medical myths because they are optimized to answer the question as framed rather than correct the underlying premise.
From Static Questions to the Patient Journey
Clinical competence is not a single decision point; it is a longitudinal process. A doctor does not simply output a diagnosis and vanish; they must gather information, coordinate care, and communicate risks over time.
To capture this, researchers are moving toward end-to-end workflow evaluations. Weixiang Yan et al. argue that evaluating models on isolated tasks provides an incomplete picture of their utility (ClinicalLab: Aligning Agents for Multi-Departmental Clinical Diagnostics in the Real World). They propose evaluating models across the entire diagnostic arc, from initial presentation to final disposition.
This longitudinal approach is exemplified by benchmarks like MedJourney, introduced by Xian Wu et al. Instead of a static question, it evaluates models across a four-stage workflow:
- Planning: Initial triage and diagnostic strategy.
- Access: Navigating insurance and administrative hurdles.
- Delivery: Executing the clinical intervention.
- Ongoing Care: Post-discharge management and patient communication.
When evaluated this way, a striking asymmetry emerges: a model that achieves near-perfect diagnostic accuracy might completely fail at the administrative navigation or patient-facing communication required to actually deliver that care.
Evaluating Non-Diagnostic Competencies
A truly versatile medical assistant must also master the administrative and interpersonal tasks that consume a clinician's day. Scott L Fleming et al. addressed this by creating MedAlign, a dataset of clinician-generated instructions paired with real Electronic Health Record (EHR) data. When forced to perform realistic tasks—like summarizing a sprawling, disorganized patient history or finding specific data points across years of unstructured notes—models exhibit high error rates that multiple-choice benchmarks completely fail to predict.
Finally, medicine requires navigating ethical boundaries and expressing empathy—dimensions entirely absent from standard biological fact retrieval. As Rui Yang et al. point out in Toward Global Large Language Models in Medicine, the historical over-reliance on simple Question Answering and Natural Language Inference tasks leaves us blind to how models handle multi-turn patient dialogues.
When we look at fields like mental health counseling, the gap becomes even more acute. The CounselingBench framework by Nguyen Viet Cuong et al. demonstrates that while frontier models possess the raw textbook knowledge to pass licensing exams, they fall significantly short of human experts when evaluated on core clinical competencies like empathy, ethical boundary management, and nuanced, multi-turn therapeutic reasoning.
By shifting the evaluation paradigm away from standardized test-taking and toward the messy, longitudinal, and interactive realities of actual healthcare, the medical AI community is beginning to build benchmarks that measure true clinical utility rather than superficial pattern matching.
Key Foundational Work
Learning to Make Rare and Complex Diagnoses With Generative AI Assistance: Qualitative Study of Popular Large Language Models (2023) — contributed 1 concept elements to this direction.
Biomedical Concept Relatedness – A large EHR-based benchmark (2020) — contributed 1 concept elements to this direction.
Evaluating large language models on a highly-specialized topic, radiation oncology physics (2023) — contributed 1 concept elements to this direction.