TL;DR
Researchers have developed a scalable "stress test" for Large Language Models (LLMs) summarizing radiology reports by using clinically grounded adversarial edits. The study reveals that while top-tier models like GPT-5-mini lead the pack, even the best models frequently miss critical clinical modifiers—such as changing "mild" to "severe"—and exhibit hallucination rates as high as 30%.
What Problem Does This Paper Address?
In the U.S., the 21st Century Cures Act has granted patients direct access to their electronic health records, including radiology reports. However, these reports are often dense with medical jargon, leading to patient anxiety and confusion. While LLMs offer a promising solution by translating these reports into "patient-friendly" summaries, they are prone to "hallucinations" and factual errors.
The core challenge is evaluation. Manual review by radiologists is the gold standard but is impossible to scale. Traditional NLP metrics like ROUGE or BERTScore often fail to capture clinical nuances—a summary might be linguistically similar to the original but clinically dangerous if it omits a "no" before "evidence of malignancy." This paper introduces a framework to automatically and robustly evaluate how well LLMs preserve clinical facts under adversarial pressure.
Methods
The authors introduced a clinically grounded adversarial framework that "stresses" LLMs using two primary strategies:
- Term Substitution: Using a radiologist-validated dictionary of 247 terms, the researchers systematically swapped words across eight semantic categories (e.g., pathology, anatomy, descriptors, and confidence terms). For example, "mild" might be replaced with "severe" to see if the LLM summary reflects this change in intensity.
- Impression Removal: The framework identifies and removes the most "keyword-dense" sentences or points from the original report's impression section to test if the LLM summary remains complete.
To evaluate the resulting summaries, the team employed an LLM-as-a-judge framework (primarily using GPT-5). This judge operates in two stages: first, it extracts clinical findings into a structured schema (Condition, Location, Severity, Certainty); second, it compares the original and adversarial versions to score the summary on Factual Faithfulness, Sensitivity to Changes, Certainty Fidelity, and Hallucination Rate.
Datasets
The study utilized a diverse corpus of 13,801 radiology report impressions sourced from three public datasets to ensure generalizability across modalities and body regions:
- Merlin: 5,125 abdominal CT reports.
- INSPECT: 3,676 chest CTPA (Computed Tomography Pulmonary Angiography) reports.
- MIMIC-IV: 5,000 MRI reports covering the spine, brain, breast, and extremities.
Key Insights and Ideas
- Nuance is the Achilles' Heel: Models were generally good at identifying primary pathologies but struggled significantly with "clinical modifiers." Failure rates were highest for anatomical modifiers (23.7% missed) and descriptors like "substantial" or "suspicious." This suggests that LLMs might downplay the urgency or specific location of a finding during simplification.
- Hallucinations are Pervasive: Even the highest-performing model, GPT-5-mini, hallucinated in roughly 28-30% of cases under term substitution. Most of these weren't fabricated diseases but rather "unsupported follow-up recommendations" or "added reasoning" not present in the original report.
- The "Judge" is Competent: The automated LLM judge showed a macro-F1 agreement score of 0.55–0.56 with human radiologists. Crucially, this was comparable to the agreement between the two human radiologists themselves (0.53), suggesting that LLM-based evaluation is a viable, scalable alternative to expert manual review.
- Prompting Matters Less Than Model Choice: While detailed prompts (assigning a "medical specialist" persona) were tested against concise ones, the choice of the underlying model had a much larger impact on robustness than the specific wording of the instructions.
Key Results
- Top Performer: GPT-5-mini achieved the highest factual faithfulness (0.851) and certainty fidelity. Claude Sonnet 4.5 followed closely, while open-source models like Llama 3.1 and Qwen 3 struggled, often scoring below 0.50 in faithfulness.
- Sensitivity Gap: Models were much better at handling "Impression Removal" (detecting that something was missing) than "Term Substitution" (detecting that a specific detail had changed). GPT-5-mini reflected changes in ~90% of removal cases but only ~40% of substitution cases.
- Failure Modes: A deep dive into specific terms showed that the word "suspicious" had a failure rate of over 50% for both GPT-5-mini and Claude, meaning the models often failed to convey the radiologist's level of concern to the patient.
Limitations and Future Work
The authors acknowledge several limitations:
- Synthetic Edits: The adversarial changes are programmed and may not capture the full, messy spectrum of real-world dictation errors or complex multi-finding inaccuracies.
- Scope: The study focused only on the "Impression" section; future work should extend this to the "Findings" section for a more comprehensive assessment.
- Patient Impact: The study evaluated clinical accuracy but did not directly measure whether patients actually understood the summaries better or if the summaries changed their clinical behavior.
- Subjectivity of Hallucinations: The low agreement between humans on what constitutes a "hallucination" highlights the inherent subjectivity in simplifying medical text, suggesting a need for better rater calibration in future studies.
Summary updated