Abstract
The Model Parameter Randomisation Test (MPRT) is highly recognised in the eXplainable Artificial Intelligence (XAI) community due to its fundamental evaluative criterion: explanations should be sensitive to the parameters of the model they seek to explain. However, recent studies have raised several methodological concerns for the empirical interpretation of MPRT. In response, we propose two modifications to the original test: Smooth MPRT and Efficient MPRT. The former reduces the impact of noise on evaluation outcomes via sampling, while the latter avoids the need for biased similarity measurements by re-interpreting the test through the increase in explanation complexity after full model randomisation. Our experiments show that these modifications enhance the metric reliability, facilitating a more trustworthy deployment of explanation methods.
Introduction
In the field of eXplainable AI (XAI), saliency maps are essential tools for interpreting model predictions by highlighting influential input regions. However, the reliability of these explanations is often questioned. The Model Parameter Randomisation Test (MPRT) was introduced as a "sanity check" to ensure that saliency maps are sensitive to model parameters rather than acting as simple visual filters, such as edge detectors. If an explanation remains unchanged after model weights are randomized, it fails the sanity check and is considered unfaithful to the model's internal logic.
Despite its importance, the original MPRT suffers from several methodological flaws. These include sensitivity to "shattering noise" in gradients, reliance on biased similarity metrics like SSIM, and inconsistent layer-wise randomization orders. This paper addresses these issues by introducing two robust alternatives: Smooth MPRT (sMPRT) and Efficient MPRT (eMPRT).
Methodology
The authors identify three critical caveats in the original MPRT: inconsistent pre-processing, the influence of bottom-up information flow during top-down randomization, and the sensitivity of similarity measures to noise. To mitigate these, the following modifications are proposed:
Smooth MPRT (sMPRT)
To address the "shattering noise" that often plagues gradient-based methods, sMPRT adopts a sampling approach similar to SmoothGrad. Instead of evaluating a single explanation, it generates $N$ perturbed versions of the input, computes their explanations, and averages them. This denoising process ensures that the evaluation reflects the true underlying attribution rather than high-frequency artifacts that might artificially lower similarity scores.
Efficient MPRT (eMPRT)
eMPRT moves away from pairwise similarity metrics, which can be biased by the visual characteristics of the explanation method. Instead, it measures the relative rise in complexity using discrete entropy ($\xi$). A faithful explanation should exhibit a significant increase in complexity (becoming more "chaotic") when the model it describes is fully randomized. The metric $\Psi_{\tau}^{\text{eMPRT}}$ quantifies this shift.
Bottom-Up Randomization
The paper advocates for randomizing layers from the input toward the output (bottom-up). This ensures that information flow is disrupted early, preventing "ghost" features from lower layers from persisting in the explanation and providing a more rigorous test of parameter sensitivity.
Experimental Results
The proposed metrics were evaluated using standard benchmarks, including ImageNet (with VGG-16 and ResNet-18) and MNIST/Fashion-MNIST (with LeNet). The researchers tested a wide array of attribution methods, including Saliency, Gradient, Integrated Gradients, SmoothGrad, and various flavors of Layer-wise Relevance Propagation (LRP).
Key Insights
- Noise as a Deceptive Signal: The original MPRT often ranked gradient-based methods as "more faithful" simply because they were noisier. When sMPRT denoises these explanations, their perceived performance drops, revealing that their sensitivity was partly an illusion caused by gradient shattering.
- Guided Backprop's Redemption: Previous studies using the original MPRT suggested Guided Backpropagation was unfaithful. However, under the more robust eMPRT metric, Guided Backprop actually performs better than many gradient-based counterparts, suggesting it was unfairly maligned by biased similarity measures.
- Metric Reliability: Using the MetaQuantus framework, the authors demonstrated that sMPRT and eMPRT are significantly more reliable and resilient to input noise than the original MPRT. For sMPRT, $N=50$ samples provide a stable, denoised quality estimate.
Conclusion
This work highlights the dangers of relying on flawed metrics to validate XAI methods. By introducing sMPRT and eMPRT, the authors provide a more trustworthy framework for assessing explanation faithfulness. While computational overhead for sampling and hyperparameter sensitivity in entropy calculations remain areas for future work, these modifications represent a significant step toward bridging the "complexity gap" between high-dimensional neural networks and human-interpretable saliency maps.
Summary updated