When we ask a standard feature attribution method to explain a neural network's decision, we typically query a single target class: "Show me the features that support the prediction of 'Cat'." The resulting heatmap usually highlights the animal's face, body, and perhaps some background texture.
However, a critical flaw emerges if we change the query to a completely different class, such as "Dog" or "Elephant". Frequently, the attribution map remains virtually unchanged, highlighting the exact same foreground pixels.
This phenomenon is known as a lack of class discriminativity. Instead of explaining the model's specific classification logic, standard attribution methods often act as generic object localizers or edge detectors, capturing the network's internal structural state rather than its decision criteria. To truly understand why a model made a specific choice, we must shift our framing from absolute attribution to contrastive feature attribution.
The Illusion of Class Sensitivity
Why do standard backpropagation and relevance-mapping techniques fail to differentiate between classes? As Leon Sixt et al. demonstrate in When Explanations Lie: Why Many Modified BP Attributions Fail, many popular modified backpropagation algorithms mathematically lose class-specific information as the signal flows backward through the deep layers of a network. The attribution signals tend to converge to a "rank-1" state that reflects low-level image structures and data distributions rather than the high-level logit differences at the output layer.
This creates a severe localization bias. For instance, Haofan Wang et al. note in Score-CAM: Score-Weighted Visual Explanations for Convolutional Neural Networks that when an image contains multiple distinct objects, traditional saliency maps struggle to isolate them based on the class query. If the network perceives a prominent foreground object, that object dominates the explanation map regardless of whether you are explaining the correct label or a completely irrelevant one.
Shifting from "Why" to "Why Not"
Neural networks are inherently discriminative models; they learn boundaries that separate classes rather than modeling the pure distribution of a single class in isolation. Therefore, an effective explanation must align with this discriminative nature. Instead of asking "Why class $A$?", we must ask a contrastive question: "Why class $A$ and not class $B$?"
The mathematical intuition behind this shift is illuminated by Yipei Wang et al. in "Why Not Other Classes?": Towards Class-Contrastive Back-Propagation Explanations. The authors show that explaining the final softmax probability of a target class is mathematically equivalent to explaining the target logit minus a weighted sum of all other logits.
$$\text{Explanation}(P_A) \propto \nabla \left( \text{logit}_A - \sum_{i \neq A} w_i \cdot \text{logit}_i \right)$$
This reveals a fundamental dichotomy in explainable AI:
- Logit-based attributions focus purely on evidence accumulation, which naturally defaults to general object localization.
- Probability-based or explicitly contrastive attributions inherently compute a delta, filtering out shared features and isolating the exact traits that drove the model to prefer one classification over its competitors.
The Subtractive Mechanism of Contrastive Attribution
The core mechanism that makes contrastive attribution work is signal subtraction. By taking the evidence supporting a target concept and subtracting the evidence supporting a dual or competing concept, the shared background noise and structural activation masks cancel out. What remains is the pure discriminative signal.
[Target Class Map (e.g., Leopard)] --> Highlights entire body, spots, background
-
[Competing Class Map (e.g., Jaguar)] --> Highlights entire body, spots, background
=
[Contrastive Saliency Map] --> Highlights only the unique rosette patterns
This mechanism is implemented across various architectures using different strategies:
- Relevance Subtraction: In Understanding Individual Decisions of CNNs via Contrastive Backpropagation, Jindong Gu et al. introduce Contrastive Layer-wise Relevance Propagation (CLRP). By explicitly subtracting the relevance of an "opposite" or distractor concept from the target class relevance and clamping negative values to zero, they eliminate the shared structural noise that plagues standard LRP.
- Comparative Gradients: Jaesik Choi et al. utilize a similar intuition in Interpreting Deep Neural Networks with Relative Sectional Propagation by Analyzing Comparative Gradients and Hostile Activations, where they contrast the gradients of a target class against "hostile" competing classes to suppress generic features.
- Logit Differencing: Instead of manipulating backpropagation rules post-hoc, Jianyang Gu et al. propose Finer-CAM: Spotting the Difference Reveals Finer Details for Visual Explanation. They calculate visual explanations based directly on the difference between the target logit and a reference logit, forcing the saliency map to bypass shared category features and expose subtle, fine-grained details.
This subtractive property is not limited to images. In the domain of multi-class time series, the DEMUX framework introduced by Ramesh Doddaiah et al. (Class-Specific Explainability for Deep Time Series Classifiers) uses a multi-objective loss function that explicitly penalizes temporal overlaps between the saliency maps of different classes. This optimization forces the explainer to find the unique temporal "fingerprints" that distinguish one sequence from another.
Contrastive Attribution in Sequence Generation
The necessity of contrastive framing becomes even more pronounced in generative autoregressive models, such as Large Language Models (LLMs). When an LLM generates a token, its absolute probability distribution is heavily influenced by syntax, grammar, and common token frequencies.
To isolate the true contextual drivers of a specific semantic choice, researchers look at the difference in logit contributions between a predicted token and a counterfactual foil token. As [Javier Ferrando et al.`]Explaining How Transformers Use Context to Build Predictions demonstrate, decomposing the final layer-wise logit contributions relative to a counterfactual alternative isolates the exact contextual features that drive a specific grammatical or semantic preference.
This approach is formalized in toolkits like Inseq, developed by Gabriele Sarti et al. (Inseq: An Interpretability Toolkit for Sequence Generation Models). By calculating feature importance relative to the difference between a chosen target token and a predefined distractor, contrastive feature attribution can pinpoint the exact input tokens responsible for biased generations or specific stylistic shifts, filtering out the massive background context shared between both potential generation paths.
Ultimately, contrastive feature attribution works because it shifts the goal of interpretability from description to discrimination. By evaluating features based on their ability to change a decision from one specific outcome to another, it strips away the shared background context and exposes the true levers of neural network decision-making.
Key Foundational Work
Understanding Individual Decisions of CNNs via Contrastive Backpropagation (2018) — contributed 3 concept elements to this direction.
Excitation Backprop for RNNs (2018) — contributed 2 concept elements to this direction.
Towards Explanation of DNN-based Prediction with Guided Feature Inversion (2018) — contributed 2 concept elements to this direction.