Aggregate Accuracy Hides Diagnosis-Specific Failure in Medical Vision-Language Models

Shreya Ajay Kale, Angad Ripudaman Singh BajwaarXiv 2026 · arXiv

TL;DR

Aggregate accuracy metrics in medical Vision-Language Models (VLMs) can mask catastrophic, diagnosis-specific failures. This paper demonstrates that several state-of-the-art models exhibit "silent blindness"—confidently denying the presence of life-threatening conditions like pneumothorax even while fluently describing their radiographic signs—despite the information being present in the models' internal representations.

What Problem Does This Paper Address?

In medical AI, we often rely on a single "headline" accuracy number to judge a model's performance. However, an average score can be deceptive. A model might achieve 90% accuracy by being perfect at identifying common conditions while completely failing to detect a rare but fatal one.

The authors investigate a phenomenon they call "whole-diagnosis collapse." They focus specifically on pneumothorax (a collapsed lung), a high-stakes diagnosis where a missed case can be fatal within hours. The danger here is twofold: not only do some models fail to see the condition, but they do so "silently." They provide fluent, medically plausible explanations for why the condition is absent, making the error incredibly difficult for a clinician to catch at the point of care.

Methods

The researchers audited five open-source medical VLMs: MedGemma-4B, Lingshu-7B, HuatuoGPT-Vision-7B, Qwen2.5-VL-7B, and MedVLM-R1.

Their methodology included several rigorous controls:

  • Prevalence-Matched Audit: They constructed a balanced dataset to ensure that high accuracy wasn't just a result of the model guessing the most common label.
  • Grounding Metric: They tested if model answers were actually tied to the image by comparing responses to original images versus "swapped" images (different patient, opposite label) and "augmented" images (slight rotations or brightness changes).
  • Linear Probing: To see if the models were truly "blind," they trained simple classifiers (linear probes) on the models' internal hidden states. This allowed them to check if the information about a diagnosis existed inside the model, even if the final text output was a "No."
  • Robustness Checks: They tested whether failures were due to prompt wording, image resolution, or model size (scaling up to 72B parameters).

Datasets

  • ChestX-ray14: The primary dataset used for the audit, consisting of over 100,000 frontal X-rays. The authors used a curated subset of 6,048 items balanced by age, sex, and diagnosis.
  • SIIM-ACR Pneumothorax: An independent validation set featuring radiologist-drawn, pixel-level masks. This was used to ensure the findings weren't just an artifact of noisy labels in the primary dataset.

Key Insights and Ideas

  • Fluent Hallucination of Absence: The most striking finding is that failing models don't just say "No"; they explain why in detail. As shown in Figure 2, a model might correctly state that a pneumothorax appears as a "clear space between the lung and chest wall" and then immediately claim that space is not present, even when it is clearly visible.
Figure 1: The same chest X-ray, four models. On a pneumothorax-positive PA film from ChestX- ray14, MedGemma and Lingshu identify the pneumothorax and describe the pleural line, while Qwen and HuatuoGPT deny it in fluent, anatomically specific language, with HuatuoGPT additionally ruling out mediastinal shift. Across all 432 pneumothorax-positive cases, Qwen and HuatuoGPT return zero positives.
  • Internal Representation vs. Output Suppression: The linear probe analysis revealed that the models actually "see" the pneumothorax. The information is decodable from early transformer layers but "fades" or is suppressed before reaching the final output layer. This suggests the failure is a logic or language-generation issue rather than a purely perceptual one.
  • Scale is Not a Panacea: Increasing the model size (e.g., Qwen2.5-VL from 7B to 72B) did not fix the pneumothorax blindness. This indicates that simply throwing more parameters at the problem won't necessarily solve specific diagnostic failures.
  • Fine-tuning is a Double-Edged Sword: Lingshu and HuatuoGPT share the same Qwen backbone. However, Lingshu (a medical fine-tune) succeeded where HuatuoGPT (another medical fine-tune) failed completely. This highlights how specific training data and alignment choices can inadvertently "break" a model's ability to report certain findings.

Key Results

  • Zero Recall: Both Qwen2.5-VL and HuatuoGPT-Vision detected zero out of 432 pneumothorax cases, effectively performing at chance level (0.500 balanced accuracy) by always answering "No."
  • Diagnosis-Specific Collapse: As shown in Table 2 (Figure 4), models that were excellent at one finding (e.g., MedGemma's 94.7% recall on Atelectasis) could be significantly worse at others (37.7% on Pneumothorax).
Table 2: Per-finding recall on 432 positive cases per finding.
  • Internal Knowledge: Despite 0% recall at the output, linear probes achieved an AUC of 0.72 to 0.75 on the SIIM-ACR dataset for the failing models. The information was there; the model just wouldn't say it.
  • Confidence vs. Accuracy: The failing models were "confidently wrong." Qwen, for instance, was 97.6% confident in its pneumothorax answers while being no better than a coin flip in reality.

Limitations and Future Work

The authors acknowledge several limitations:

  • Label Noise: ChestX-ray14 labels are derived from radiology reports using NLP, which can be imperfect. While the SIIM-ACR replication mitigated this, it remains a factor in large-scale medical AI research.
  • Scope: The study focused on binary "Yes/No" questions rather than the complex, free-form reports radiologists actually write.
  • Demographics: While they balanced for age and sex, other demographic factors (like race or socioeconomic status) were not available in these datasets.

Future Work: The authors suggest that because the information is present internally, the failure might be addressable at the output stage (e.g., through better alignment or targeted interventions) rather than requiring a complete overhaul of the model's vision system. They also call for a shift in how we evaluate medical AI: moving away from aggregate scores toward mandatory per-finding reporting on balanced datasets.

Summary updated

Abstract

Medical vision-language models (VLMs) are compared mainly by accuracy averaged across findings, a summary that can hide complete failure on a single diagnosis. We audit five open medical VLMs (MedGemma-4B, Lingshu-7B, HuatuoGPT-Vision-7B, Qwen2.5-VL-7B, and MedVLM-R1) on 6,048 ChestX-ray14 questions balanced by sex, age, finding, and label. On the 432 pneumothorax-positive X-rays, HuatuoGPT-Vision and Qwen2.5-VL detect zero cases while MedGemma and Lingshu detect 38% and 67% of the same images. The two failing models answer "no" to all 864 pneumothorax cases (balanced accuracy 0.500, chance), yet produce fluent, anatomically specific explanations that name the signs of a collapsed lung and then deny them, with no hedge or abstention. This failure is not an artifact of prompting, image resolution, label ambiguity, or model scale. It persists across four prompt formats, at native resolution, and on single-label cases. A within-model control shows the same forced prompt gives HuatuoGPT 79% recall on nodule but only 6% on pneumothorax. It survives a tenfold scale increase (Qwen2.5-VL-72B detects 4%) and replicates on SIIM-ACR, an independent dataset with radiologist-drawn masks, where the failing models detect 0 of 432 confirmed pneumothoraces, including 0 of the 144 largest lesions. Reading the models' logits and internal states shows this is not blindness. The two failing models are highly confident yet no better than chance and a linear probe recovers pneumothorax from their hidden representations even where recall is zero, so the finding is represented internally but suppressed at the output. Aggregate accuracy therefore cannot establish that a medical VLM can see a given finding, because a model that misses one can still describe it convincingly. Reporting per-finding recall and specificity on balanced cases is needed to tell the difference.
BibTeX
@misc{kale2026aggregate,
  author = {Shreya Ajay Kale and Angad Ripudaman Singh Bajwa},
  title = {{Aggregate Accuracy Hides Diagnosis-Specific Failure in Medical Vision-Language Models}},
  year = {2026},
  eprint = {2611.14975},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.14975},
}
Figure