Introduction
Saliency mapping is a cornerstone of explainable AI (XAI), providing heatmaps that highlight influential pixels in a model's prediction. However, traditional methods are fundamentally input-specific: they explain why a model classified a particular image as a "Panda," rather than what the model understands about the "Panda" category as a whole. This dependency often leads to maps that merely highlight object boundaries, failing to capture the model's internal logic or detect hidden vulnerabilities like backdoors.
This paper introduces a novel input-agnostic saliency mapping technique. By decoupling the visualization from specific input images, the method reveals the high-level geometric features a model associates with a learned concept, offering a more holistic interpretation of deep visual classifiers.
The Input-Agnostic Approach
The core limitation of conventional saliency is its "weak dependence" on the classifier. If a method produces similar maps for both a flawed model and a perfect theoretical one, it isn't truly explaining the specific model's behavior. Input-agnostic mapping addresses this by generating visualizations directly from the model's internal gradients without requiring a reference dataset.
Methodology
The proposed algorithm iteratively constructs a map from a neutral "seed" (e.g., a black image) through three main stages:
- Gradient Nudging: Random data points are "nudged" toward the local minima of the model’s loss surface for a target class.
- Integration and Projection: These gradient directions are accumulated and projected onto a bounded $\ell_2$-ball. This constraint forces disparate features to "collaborate," resulting in coherent geometric patterns rather than isolated high-intensity pixels.
- Refinement: To suppress non-perceptual noise, a weighting matrix (inspired by GradCAM) is applied, highlighting the structural features the model prioritizes.
Experimental Results
The method was evaluated on VGG-16, ResNet-50, and DenseNet-121 architectures pre-trained on ImageNet.
Qualitative Analysis
The generated maps for various ImageNet classes demonstrate that the model learns distinct, human-interpretable textures and shapes. For instance, the "Honeycomb" map captures hexagonal patterns, while the "Mosque" map visualizes dome-like structures.
Quantitative Fidelity
To measure how well these maps represent generic features, the authors introduced the Model-score ($M_{score}$). A map generated for a specific model (e.g., VGG-16) is fed into other classifiers. The results showed high cross-architecture consistency:
- Intra-architecture: VGG-16 maps achieved a score of 0.99 when tested on other VGG-16 instances.
- Inter-architecture: Maps from ResNet-50 maintained high scores (~0.94) when evaluated by DenseNet-121, suggesting that different models converge on similar geometric concepts.
Security Applications: Backdoor Detection
A significant advantage of input-agnostic mapping is its ability to expose "Trojan" backdoors. In compromised models, the map for a target class will often reconstruct the hidden trigger pattern (e.g., a specific logo) used to fool the classifier. This allows for the detection of malicious triggers without prior knowledge of their appearance, a task nearly impossible with standard input-specific saliency.
Conclusion and Future Work
Input-agnostic saliency mapping provides a powerful new lens for auditing deep learning models. While the current refinement stage uses external operators and the computational cost is higher than traditional methods (14–19 minutes per map), the ability to visualize "pure" model concepts and detect security threats represents a major advancement in model interpretability. Future research will focus on identifying non-visible adversarial triggers and further reducing reliance on external refinement operators.
Summary updated