This paper investigates the "black box" of Vision-Language Models (VLMs) in the high-stakes domain of mammography. The authors move beyond simple accuracy to evaluate coherence: do a model's answer, its stated confidence, and its written explanation actually align, and are they grounded in visual evidence?
TL;DR
Researchers found that mammography VLMs often exhibit "mirage reasoning," where they provide confident, detailed explanations that contradict their final answers or persist even when the medical image is removed. The study reveals that these models are highly sensitive to superficial changes like option ordering, suggesting that current medical VLM performance is often anchored in linguistic patterns rather than genuine clinical understanding.
What Problem Does This Paper Address?
As VLMs are increasingly proposed for autonomous medical tasks—like triaging mammograms—evaluators typically rely on accuracy metrics. However, in medicine, being "right for the wrong reasons" is dangerous. A model might correctly label a case as BI-RADS 0 (needs further imaging) but its explanation might describe the breast as "normal" (which should be BI-RADS 1).
The authors address the lack of multi-signal evaluation. They argue that for a clinician to trust an AI, the model’s three primary outputs—the answer, the confidence score, and the rationale—must be internally consistent and externally grounded in the image data.
Methods
The researchers evaluated four open-source VLMs: two medically-adapted (MedGemma-4B, Lingshu-7B) and two general-purpose (Qwen2-VL-7B, InternVL3-8B).
They employed a novel evaluation framework involving:
- Joint Signal Elicitation: Using a structured prompt to force the model to provide an answer, a confidence score for every option (not just the chosen one), and a one-sentence explanation.
- Controlled Perturbations: To test visual grounding and elicitation robustness, they subjected the models to six conditions:
- Original: The baseline image and question.
- Visual Perturbations: Replacing the mammogram with a blank image, an unrelated photo (e.g., a pastry), or removing the image entirely.
- Linguistic Perturbations: Shuffling the order of multiple-choice options and slightly rewording the question.
- Coherence Metrics: They developed a rule-based matcher to extract the "implied answer" from the natural language explanation to see if it matched the selected multiple-choice label.
Datasets
The study utilized two major mammography benchmarks from the MammoVQA suite:
- RSNA Screening Mammography: Focused on BI-RADS assessment and breast density (ACR categories).
- EMBED: A large-scale, racially diverse dataset used for exam-level BI-RADS categorization.
Key Insights and Ideas
- Confidence-Evidence Inversion: Ideally, a model’s confidence should drop when the image is removed. Instead, models like MedGemma remained highly confident (~95%) in a specific category even when looking at a blank gray square.
- The "None of the Above" (NOTA) Failure: When images were replaced with junk data, a grounded model should select NOTA. Most models failed this; MedGemma, for instance, selected a clinical category 100% of the time for unrelated photos, showing a total lack of "epistemic humility."
- Option-Order Fragility: Simply shuffling the order of A, B, C, and D flipped the models' answers in 40% to 99% of cases. This suggests the models are "multiple-choice selectors" rather than "image interpreters."
- Self-Contradictory Rationale: The authors found cases where a model's reasoning was clinically sound but its final answer was wrong, or vice versa. Accuracy metrics would credit a model for a correct label even if the reasoning proved it didn't understand the image.
Key Results
The quantitative results paint a sobering picture of current VLM capabilities in specialized medical tasks:
- Low Baseline Accuracy: On BI-RADS tasks, no model exceeded the majority-class prior (the accuracy you'd get by just guessing the most common label).
- High Flip Rates: Under the "Perturbed-Options" condition, MedGemma's accuracy dropped by 24.8 percentage points on BI-RADS just by reordering the choices.
- Reasoning Disconnect: In the EMBED dataset, Lingshu-7B's explanation named the correct category 57.6% of the time, yet it only selected the correct multiple-choice answer 10.3% of the time.
Limitations and Future Work
The authors acknowledge several limitations:
- Format Constraints: The study is limited to multiple-choice questions; free-text report generation might yield different coherence patterns.
- Model Scope: Only open-weights models in the 4-8B parameter range were tested. The behavior of larger, closed models (like GPT-4o) remains unknown in this specific mammography context.
- Clinical Realism: The perturbations did not include clinically nuanced changes like swapping laterality (left vs. right breast) or comparing against prior images, which are standard in radiologist workflows.
Future work should explore whether domain-specific fine-tuning can actually fix these coherence failures or if they are fundamental to the architecture of current vision-language transformers.
Summary updated