Abstract
In recent years, the interpretability of Deep Neural Networks (DNNs) has garnered significant attention, particularly due to their widespread deployment in critical domains like healthcare, finance, and autonomous systems. To address the challenge of understanding how DNNs make decisions, Explainable AI (XAI) methods, such as saliency maps, have been developed to provide insights into the inner workings of these models. This paper introduces DiffCAM, a novel XAI method designed to overcome limitations in existing Class Activation Map (CAM)-based techniques, which often rely on decision boundary gradients to estimate feature importance. DiffCAM differentiates itself by considering the actual data distribution of the reference class, identifying feature importance based on how a target example differs from reference examples. This approach captures the most discriminative features without relying on decision boundaries or prediction results, making DiffCAM applicable to a broader range of models, including foundation models. Through extensive experiments, we demonstrate the superior performance and flexibility of DiffCAM in providing meaningful explanations across diverse datasets and scenarios.
Introduction
Deep Neural Networks (DNNs) are often criticized as "black boxes," making it difficult to trust their decisions in high-stakes environments. Saliency maps—visualizations that highlight influential pixels—are a primary tool for opening these boxes. However, popular methods like GradCAM and ScoreCAM often focus on the model's internal decision boundaries rather than the underlying data distribution. This paper introduces DiffCAM, a data-driven approach that shifts the focus to actual feature differences between a target image and a reference group.
The Core Problem: Boundaries vs. Distributions
Traditional gradient-based methods identify the "fastest" way to change a model's prediction. While effective for some tasks, this doesn't always align with human intuition.
Consider a student who fails a test despite a high writing score because their reading score was exceptionally low. A gradient-based model might highlight "writing" as the most important feature if writing is generally a stronger predictor for the class. However, the true reason for failure in this specific instance is the reading score. DiffCAM addresses this by calculating how an individual data point deviates from a "Pass" distribution, rather than just following the model's internal weights.
The DiffCAM Framework
DiffCAM operates within the general Class Activation Map (CAM) framework, where a saliency map is generated as a weighted sum of feature maps. The innovation lies in how these weights ($w$) are calculated:
- Feature Extraction: The system extracts deep embeddings ($z$) for both the target image and a selected reference group.
- Optimization: DiffCAM seeks a direction in feature space that maximizes the distance between the target image and the reference group's mean, while minimizing the variance within that reference group.
- Mathematical Solution: The optimal weight vector is derived as $w^* = S_v^{-1}(z - \hat{\mu})$, where $S_v$ is the covariance matrix of the reference group and $\hat{\mu}$ is its mean.
This approach provides unprecedented flexibility. By simply changing the reference group, users can ask different questions of the model: "Why is this a 6?", "Why is this not a 4?", or "What makes this specific 6 unusual?"
Experimental Evaluation
The authors evaluated DiffCAM across diverse benchmarks to test its robustness and faithfulness:
- ImageNet: Used to measure object localization accuracy. DiffCAM achieved an AUPRC of 0.741, outperforming GradCAM (0.727) and ScoreCAM (0.709).
- Medical Imaging (RSNA & SIIM-ACR): In pneumonia detection tasks, DiffCAM significantly outperformed baselines in locating abnormalities, achieving an AUPRC of 0.492 compared to GradCAM's 0.467.
- CUB-200-2011: On this fine-grained bird dataset, DiffCAM demonstrated superior precision in counterfactual explanations, identifying the specific traits that distinguish one species from another.
Key Insights
- Comprehensive Patterns: By focusing on population differences, DiffCAM captures more complete object patterns than gradient methods, which often focus too narrowly on the most discriminative edges.
- Counterfactuals "For Free": Unlike other methods that require GANs to generate synthetic images, DiffCAM provides counterfactual insights simply by using a different class as the reference group.
- Model Agnostic: Because it does not rely on backpropagating gradients, DiffCAM is compatible with a wide range of architectures, including Vision Transformers (ViTs) and Self-Supervised Learning (SSL) models.
Conclusion
DiffCAM represents a significant step forward in XAI by grounding explanations in data distributions rather than just model internals. While it currently assumes a linear relationship in the final feature layer, its ability to provide flexible, intuitive, and highly accurate saliency maps makes it a powerful tool for interpreting complex deep learning models in both general and specialized domains.
Summary updated