Introduction
Deep learning (DL) models are often criticized as "black boxes," where local explanations like saliency maps provide insights into individual predictions but fail to capture the model's general behavior across a dataset. This lack of global explainability is particularly problematic in medical imaging, where models might exploit "shortcuts"—such as data artifacts or camera-specific watermarks—rather than learning clinically relevant features. This paper introduces a framework for global explainability tailored to aligned image datasets, common in fields like ophthalmology and radiology.
Methods: Global Explainability Framework
The proposed framework leverages the inherent alignment in medical imaging, where specific pixel coordinates consistently correspond to the same anatomical structures across different subjects.
Pixel-wise Aggregation
The researchers generate image-wise explanations using GradCAM for all true positive predictions. These heatmaps are then averaged and weighted by the model's confidence to produce a "Global Explanation" map for each disease label.
PEPPR Validation
To ensure these global maps faithfully represent the model's logic, the authors developed PEPPR (Progressive Erasing Plus Progressive Restoration).
- Progressive Erasing: Pixels are removed starting from the least important according to the global map. A robust model should maintain high performance until only critical regions remain.
- Progressive Restoration: Starting from a blank image, pixels are added back based on importance. A sharp performance increase upon restoring "important" regions confirms the model's reliance on those areas.
Datasets and Pathologies
The study utilized the Tsukazaki Optos Public (TOP) dataset, comprising 13,047 ultra-widefield retinal images. The focus was on three conditions with distinct spatial characteristics:
- Diabetic Retinopathy: Pathology distributed across the retina.
- Glaucoma: Localized near the optic disc.
- Retinal Detachment: Often appearing in specific quadrants (e.g., upper-right).
Key Results and Insights
The ResNet18 model demonstrated high diagnostic performance, with AUCs of 0.98 for Retinal Detachment, 0.93 for Diabetic Retinopathy, and 0.92 for Glaucoma.
Information Density and Robustness
PEPPR analysis revealed significant information density. For Glaucoma, the model maintained an AUC > 0.80 using only 5% of the image area surrounding the optic disc. Furthermore, erasing 50% of the "least important" pixels resulted in negligible performance drops, validating the global explanation's ability to identify diagnostic regions.
Shortcut Detection
Global maps for Glaucoma indicated a focus on image edges. While clinically unexpected, PEPPR showed the model remained functional even when these edges were removed, suggesting the model was robust despite picking up peripheral artifacts.
Conclusion
By utilizing the "superpower" of image alignment, this framework transforms simple averaging into a rigorous diagnostic tool for DL models. The two-way validation provided by PEPPR ensures that global explanations are not merely visual artifacts but accurate maps of the model's decision-making process, paving the way for more transparent and trustworthy AI in healthcare.
Summary updated