TL;DR
This paper introduces a novel dual-framework, EGxFSL and xGAL, to tackle the critical challenges of limited labeled data and lack of interpretability in medical image analysis. By integrating expert knowledge into both model training and data acquisition, the framework achieves high accuracy and clinically relevant explanations, even with minimal data.
What Problem Does This Paper Address?
Medical image analysis has seen tremendous progress thanks to deep learning, but its widespread clinical adoption is hampered by two major hurdles: the scarcity of high-quality labeled data and the "black box" nature of deep learning models, which lack interpretability. Few-shot learning (FSL) offers a solution to data scarcity by enabling models to learn from very few examples, but these models often lack transparency. Active learning (AL) methods aim to reduce annotation costs by intelligently selecting samples for labeling, but they typically overlook the interpretability of the selected data. Existing explainable AI (xAI) techniques, while providing insights into model decisions, are often applied post-hoc and don't influence the learning process itself, potentially leading to models relying on spurious correlations. This paper addresses the need for a unified approach that simultaneously improves data efficiency, classification accuracy, and clinical interpretability in medical imaging.
Methods
The authors propose a two-pronged approach:
- Expert-Guided Explainable Few-Shot Learning (EGxFSL): This component integrates radiologist-provided regions of interest (ROIs) directly into the FSL training process. It uses a prototypical network as the base architecture and incorporates an "explanation alignment loss" based on Grad-CAM heatmaps and expert ROIs. This loss, combined with the standard classification loss, encourages the model to focus its attention on diagnostically relevant areas, ensuring that its predictions are not only accurate but also interpretable.
- Explainability-Guided Active Learning (xGAL): This component builds upon EGxFSL to intelligently select unlabeled samples for annotation. Instead of relying solely on predictive uncertainty (a common AL strategy), xGAL considers both classification uncertainty and "explanation misalignment" (how well the model's attention aligns with expert annotations). By prioritizing samples that are both uncertain and have misaligned explanations, xGAL aims to maximize the improvement in both accuracy and interpretability with each newly labeled sample. This creates a closed-loop system where explainability guides data acquisition, and the acquired data further refines the model's explainability.
Datasets
The framework was evaluated on three publicly available, expert-annotated medical imaging datasets:
- BraTS (MRI): A multimodal brain tumor dataset with segmentation masks.
- VinDr-CXR (Chest X-ray): A large-scale chest X-ray dataset with bounding boxes marking thoracic abnormalities.
- SIIM-FISABIO-RSNA-COVID-19 (Chest X-ray): A chest X-ray dataset annotated for COVID-19 related abnormalities.
The framework was also validated on a breast ultrasound dataset (BUSI) to demonstrate cross-modality applicability.
Key Insights and Ideas
- Explainability as a Guiding Principle: The core idea is that interpretability isn't just a post-hoc analysis tool but a crucial component that should actively guide the learning process, especially in data-scarce medical scenarios.
- Synergy between FSL, AL, and xAI: The paper demonstrates a powerful synergy by combining these three fields. FSL provides data efficiency, AL optimizes annotation cost, and xAI ensures clinical relevance and trust.
- Beyond Uncertainty in AL: Traditional AL methods often rely too heavily on predictive uncertainty, which can be unreliable in deep models. Incorporating explanation misalignment provides a more robust signal for selecting informative samples.
- Probabilistic Supervision for Noisy Annotations: Recognizing that expert annotations can be variable, the framework treats them as probabilistic guidance rather than absolute ground truth, making it more robust to real-world annotation imperfections.
- Cross-Modality Generalization: The success on breast ultrasound suggests the framework's principles are applicable beyond MRI and X-ray imaging.
Key Results
- EGxFSL Performance: The guided EGxFSL model consistently outperformed non-guided baselines across all datasets and shot configurations. For example, on BraTS, it achieved 92.05% accuracy (vs. 89.21% for baseline), on VinDr-CXR, 76.12% accuracy (vs. 56.21%), and on SIIM-COVID, 62.08% accuracy (vs. 51.00%).
- xGAL Data Efficiency: Under severe data constraints, xGAL achieved 76.85% accuracy on BraTS with only 680 labeled samples, significantly outperforming random sampling (58.01%). Similar improvements were seen on other datasets.
- Improved Interpretability: Grad-CAM visualizations showed that the guided models consistently focused on diagnostically relevant regions, aligning with expert annotations, while non-guided models often highlighted irrelevant areas.
- Optimal Explanation Weight: The study found that a moderate explanation weight (α = 0.10) was crucial for balancing classification accuracy and attention alignment, with both under- and over-emphasis leading to performance degradation.
Limitations and Future Work
- Dataset Limitations: Evaluation was primarily limited to MRI and chest X-ray, with preliminary validation on ultrasound.
- Annotation Granularity: Performance was stronger with fine-grained segmentation masks compared to coarse bounding boxes, suggesting annotation detail impacts alignment effectiveness.
- Grad-CAM Resolution: Grad-CAM has inherent resolution constraints and can be noisy in few-shot settings.
- Requirement for Spatial Annotations: The framework relies on existing spatial annotations.
- Hyperparameter Adaptation: Fixed hyperparameters might need adjustment for new clinical scenarios.
Future work includes expanding evaluation to more modalities, investigating the impact of annotation granularity, integrating multi-expert annotations, developing adaptive hyperparameter selection, and conducting prospective clinical validation.
Summary updated