Abstract
Saliency maps are essential tools for interpreting neural network decisions, yet they are often criticized for their instability—small changes in training data can lead to vastly different explanations. This paper provides a mathematical foundation for understanding how Gaussian smoothing, as implemented in algorithms like Smooth-Grad, stabilizes these maps. We prove a fundamental trade-off: while increasing smoothing improves stability across different training runs, it simultaneously reduces the "fidelity" of the explanation to the model's local decision logic.
Introduction
In high-stakes domains such as medical imaging, the reliability of AI explanations is paramount. If two identical neural network architectures trained on slightly different subsets of the same data produce conflicting saliency maps for the same input, the trust in these "phenomena discovery" tools is undermined. This "jitteriness" is a well-known issue in gradient-based interpretability.
While the Smooth-Grad algorithm—which averages gradients over multiple noisy versions of an input—is a popular remedy for visual noise, its theoretical properties have remained largely unquantified. This work bridges the gap between empirical intuition and mathematical rigor by analyzing the relationship between smoothing, stability, and fidelity.
Theoretical Framework
The authors utilize the algorithmic stability framework to quantify how an explanation changes when a single training sample is modified. Key theoretical contributions include:
- Stability Bounds: Mathematical upper bounds were established for Simple-Grad, Integrated-Gradients, and Smooth-Grad.
- The $1/\sigma$ Factor: The analysis proves that Smooth-Grad’s stability error improves proportionally to $1/\sigma$, where $\sigma$ represents the noise level. Higher noise levels effectively "wash out" the instabilities inherent in individual training runs.
- Fidelity Error: Fidelity is defined as the distance between the smoothed saliency map and the original local gradient. The paper proves that this error grows linearly with the smoothing parameter $\sigma$.
Experimental Results
The theoretical bounds were validated using standard benchmarks, including CIFAR-10 (ResNet34) and ImageNet (ResNet50, Swin Transformer, ConvNeXt-T). To simulate training randomness, the researchers split ImageNet into disjoint sets of 500,000 samples, comparing the resulting saliency maps.
The Stability-Fidelity Trade-Off
The core finding is a necessary trade-off: an explanation cannot be perfectly stable across training runs while remaining perfectly faithful to a specific model's raw local gradient. As $\sigma$ increases, the saliency maps transition from "pixel-perfect" but noisy representations to more stable "blobs" that highlight general regions of interest.
Smoothing as a Regularizer
Gaussian smoothing acts as a mathematical filter, removing high-frequency noise caused by stochastic training elements like weight initialization and batch ordering. This regularization makes the explanations more consistent across different models, even those with different architectures (e.g., ResNet vs. Swin Transformer).
Discussion and Conclusion
The choice of smoothing level should be dictated by the application. For debugging a specific model's failure, high fidelity (low $\sigma$) is preferred to capture precise local behavior. Conversely, for scientific discovery, where the goal is to identify general rules within the data, high stability (high $\sigma$) is more valuable.
Future work should investigate whether these trade-offs extend to perturbation-based or CAM-based methods and ensure that stabilized maps still pass essential "sanity checks," such as sensitivity to model weight randomization.
Summary updated