This study, published in Scientific Reports, explores how the presentation of AI-generated advice—specifically its "explainability" and its purported source—affects the diagnostic performance of physicians with varying levels of expertise.
TL;DR
Explainable AI (XAI) acts as a powerful equalizer in medical imaging: non-radiologist physicians see a significant boost in diagnostic accuracy when AI advice includes visual annotations, nearly matching the performance of specialists. Interestingly, physicians showed "algorithmic appreciation," performing better and expressing higher confidence when advice was labeled as coming from an AI rather than a human colleague.
What Problem Does This Paper Address?
While AI models often outperform humans in isolated radiology benchmarks, their real-world impact depends on the "human-in-the-loop" clinical setting. There is a significant gap in our understanding of how physicians actually weigh AI advice.
The authors address three specific questions:
- Does providing a visual "explanation" (arrows on an X-ray) improve a doctor's diagnosis?
- Does the "source" label (telling a doctor the advice is from an AI vs. a human expert) change how they use that advice?
- Do these effects differ between "task experts" (radiologists) and "non-task experts" (internal or emergency medicine physicians)?
Methods
The researchers employed a 2 (Explainability: annotated vs. non-annotated) × 2 (Source: AI vs. human) mixed factorial design.
- Participants: 223 physicians (117 non-task experts in IM/EM and 106 task experts in Radiology).
- The Task: Participants reviewed eight chest X-ray cases using a professional-grade DICOM viewer. For each case, they received "advice" (findings and a diagnosis) that was always correct.
- The Manipulations:
- Explainability: Half the cases included visual annotations (arrows) pointing to the pathology; the other half did not.
- Source: Participants were told the advice came from either "CHEST-AI" (a deep-learning model) or "Dr. S. Johnson" (an experienced radiologist).
- Metrics: The primary dependent variables were diagnostic accuracy, advice quality ratings, and diagnostic confidence.
Datasets
The study utilized the MIMIC Chest X-ray (MIMIC-CXR) database, an open-source repository of de-identified radiographs. A panel of senior radiologists curated eight specific cases that reflected everyday clinical practice and common diagnostic pitfalls (e.g., pneumothorax, enlarged heart).
Key Insights and Ideas
- XAI as an Equalizer: The most striking finding was that non-task experts (IM/EM physicians) receiving annotated AI advice achieved an accuracy of 86.84%. This put them nearly on par with radiologists receiving standard human advice (88.00%). This suggests XAI can bridge the expertise gap in settings where specialists aren't immediately available.
- Algorithmic Appreciation over Aversion: Contrary to some psychological theories suggesting humans are biased against algorithms, these physicians were more accurate when they believed the advice came from an AI.
- The "Second Opinion" Confidence Boost: Radiologists didn't necessarily become more accurate with AI advice (likely due to a "ceiling effect" where they were already performing at peak), but their confidence spiked when the AI agreed with them. The authors suggest experts may view AI as a valuable, independent validation tool that uses a different "logic" than a human peer.
- The Value of "Where": For non-experts, the written diagnosis wasn't enough; they needed to see where the AI was looking to fully trust and integrate the advice.
Key Results
- Accuracy: Receiving annotated advice from an AI resulted in the highest overall diagnostic accuracy. For non-experts, annotations improved performance by 5.66% (p=0.042).
- Quality Ratings: Non-experts rated the quality of annotated advice 0.24 points higher on a 7-point scale compared to non-annotated advice.
- Confidence: Radiologists reported significantly higher confidence (+0.32 points) when advice was labeled as AI-generated (p < 0.0001), whereas non-experts' confidence remained largely unaffected by the source label.
- AI Knowledge: Higher self-reported AI knowledge was a significant predictor of better task performance and higher confidence.
Limitations and Future Work
- The "Perfect AI" Bias: A major limitation is that the study only provided correct advice. In reality, AI makes mistakes. Future research must investigate if "explainable" advice makes it harder for doctors to spot AI errors (a phenomenon known as over-reliance).
- Sample Size and Scope: The study used a small set of cases (8 total, with only 6 used in final analysis) and was conducted in an online experimental setup rather than a high-pressure clinical environment.
- Ceiling Effects: Because radiologists are highly skilled, the study struggled to measure a statistically significant accuracy "boost" for them, even if one existed.
- Future Direction: The authors suggest exploring "reading time" as a metric—does XAI make doctors faster, or does it lead to "lazy" viewing?
Summary updated