Abstract
Artificial intelligence has become pervasive across disciplines and fields, and biomedical image and signal processing is no exception. The growing and widespread interest on the topic has triggered a vast research activity that is reflected in an exponential research effort. Through study of massive and diverse biomedical data, machine and deep learning models have revolutionized various tasks such as modeling, segmentation, registration, classification and synthesis, outperforming traditional techniques. However, the difficulty in translating the results into biologically/clinically interpretable information is preventing their full exploitation in the field. Explainable AI (XAI) attempts to fill this translational gap by providing means to make the models interpretable and providing explanations. Different solutions have been proposed so far and are gaining increasing interest from the community. This paper aims at providing an overview on XAI in biomedical data processing and points to an upcoming Special Issue on Deep Learning in Biomedical Image and Signal Processing of the IEEE Signal Processing Magazine.
Introduction: The Translational Gap
Deep learning (DL) has fundamentally changed the landscape of biomedical image and signal processing. From automated segmentation of 3D MRI scans to the classification of complex genomic sequences, DL models often outperform traditional techniques. However, their "black-box" nature creates a significant "translational gap."
In a clinical setting, a high-accuracy prediction is rarely enough. If an AI flags a scan for a potential tumor, a clinician needs to understand why that decision was reached to trust and act upon it. This lack of transparency actively slows the integration of AI into actual patient care. Explainable AI (XAI) serves as the bridge, aiming to create "human-centric" intelligent systems that are interpretable, trustable, and medically plausible.
A Taxonomy of Explainable AI
XAI approaches in the biomedical field are generally categorized into two main frameworks:
1. Ante-hoc Models
These models are "interpretable by design." Examples include linear regression, decision trees, and rule-based systems. While their logic is transparent from the outset, these models often lack the predictive power required to process complex, multi-modal, and high-dimensional biomedical data.
2. Post-hoc Methods
These techniques attempt to provide explanations for pre-trained, complex "black-box" models. Key methodologies include:
- Feature Probing: Methods like SHAP (SHapley Additive exPlanation) and LIME (Local Interpretable Model-Agnostic Explanations) quantify the contribution of specific features to a model's decision.
- Saliency and Gradient-based Maps: Techniques such as Grad-CAM, SmoothGrad, and Layer-wise Relevance Propagation (LRP) generate heatmaps. In medical imaging, these maps highlight the specific anatomical regions (e.g., a specific lung segment in a CT scan) that most influenced the model’s output.
- Graph-based XAI: Frameworks like the Cell Graph ATtention Network (CGAT) use attention mechanisms to identify relevant nodes—such as specific cell types or clusters—within tissue images.
Clinical Case Studies
The efficacy of XAI is best demonstrated through its application to diverse medical challenges, where it validates AI outcomes against known biological priors.
Neuroimaging and Brain Aging
Predicting "Brain Age" (BA) from T1-weighted MRI scans helps researchers understand the difference between chronological age and biological brain health. Using Grad-CAM, researchers have successfully identified specific brain regions that correlate with the "brain age delta," helping to disentangle normal aging from disease-specific neurodegeneration.
COVID-19 Diagnosis and Multi-Domain Shift
A major hurdle in AI-driven COVID-19 diagnosis is "multi-domain shift"—the variation in image characteristics across different hospital scanners. The CIFD-Net model utilized XAI to ensure the AI was focusing on infected lung tissue rather than hardware-specific artifacts. By visualizing activation regions, radiologists could confirm the model's clinical validity across different centers.
Oncology and the Tumor Microenvironment
In pancreatic cancer research, the CGAT framework was applied to multiplex immunofluorescence (mIF) images. The XAI component identified "invasive margins" (the interface between tumor and stromal regions) as the most predictive feature for disease grading, providing a biologically grounded explanation for the model's assessment.
Challenges and Future Directions
Despite its promise, several hurdles remain for the full exploitation of XAI in medicine:
- The Validation Hurdle: Unlike classification accuracy, there is often no "ground truth" for an explanation. Proving that an AI's reasoning is biologically correct, rather than just a correlation, remains a significant challenge.
- The Performance-Explainability Trade-off: There is a persistent tension between the accuracy of complex DL models and the simplicity required for human interpretability. Finding the "sweet spot" is a primary research goal.
- Domain Priors and Bias: For XAI to be useful, explanations must be medically plausible. XAI must also be robust enough to flag "confounders," such as a model learning to identify a disease based on the type of hospital bed in the background of an image.
- Human-in-the-loop: Future work must focus on "application-grounded" validation, where doctors are part of the evaluation process to ensure explanations are useful in a clinical workflow.
Conclusion
The transition from "black-box" AI to transparent, explainable systems is essential for the future of digital health. By incorporating domain priors and providing uncertainty quantification, XAI can transform AI from an opaque tool into a collaborative partner in clinical decision-making.
Summary updated