Stabilizing Stochastic Explanations for Reliable Interpretations

You are reading version 1 of this direction, frozen 2026-10-04. The latest version is version 2.

Version 1 of 2285 papers394 conceptsfrozen 2026-10-04

When a machine learning model makes a high-stakes decision—diagnosing a disease, detecting a mechanical fault, or flagging a security vulnerability—we often ask "why?" Post-hoc explanation methods like SHAP, LIME, and Integrated Gradients are designed to answer this by attributing the decision to specific input features.

However, a growing body of research reveals a troubling reality: these explanations are often "fickle." If you retrain the same model architecture on the same data, or even just run the explanation algorithm twice, you might get two entirely different justifications for the same prediction. This instability undermines the very trust these methods were meant to build.

The Sources of Interpretive Noise

The inconsistency in model explanations stems from two primary levels of stochasticity: the training process and the explanation mechanism itself.

Model-Level Instability
Even with identical architectures and datasets, neural networks are sensitive to random seeds. Fabian Eitel and Kerstin Ritter demonstrated this in Testing the Robustness of Attribution Methods for Convolutional Neural Networks in MRI-Based Alzheimer's Disease Classification. By retraining a network ten times, they found that while some methods like Layer-wise Relevance Propagation (LRP) remained consistent, others produced wildly different heatmaps. This suggests that many "explanations" are actually capturing artifacts of a specific local minimum or random weight initialization rather than a fundamental property of the data.

Algorithm-Level Instability
Many popular Explainable AI (XAI) tools are themselves stochastic. Methods like LIME and SHAP often rely on random perturbations or sampling to approximate a model's local behavior. Jonas Schulz and Rafael Poyiadzi note in Uncertainty Quantification of Surrogate Explanations: an Ordinal Consensus Approach that these random samples introduce high variance. If the sampling budget is too low, the resulting feature rankings can fluctuate significantly between runs, making the explanation a product of "sampling noise" rather than model logic.

Why Stability Matters for Trust

In low-stakes environments, a jittery explanation is a minor nuisance. In high-stakes domains, it is a dealbreaker.

In healthcare, Shivapratap Gopakumar and colleagues argue that if the variables identified as "most important" for a patient change every time the model is updated, clinicians will perceive the system as unreliable (Stabilizing High-Dimensional Prediction Models Using Feature Graphs). Similarly, in industrial monitoring, Lucas Costa Brito highlights that inconsistent diagnostic results prevent engineers from performing reliable root cause analysis on rotating machinery (An explainable artificial intelligence approach for unsupervised fault detection and diagnosis in rotating machinery).

Strategies for Stabilization

To move beyond "noisy" explanations, researchers are developing several strategies to enforce consistency.

1. Amortization and Learning to Explain

Instead of sampling at inference time, which is slow and stochastic, researchers like Chenghao Yang and He He propose amortized explanations. In Efficient Shapley Values Estimation by Amortization for Text Classification, they train a separate model to directly predict Shapley values. This turns a stochastic sampling process into a deterministic function, providing stable and faithful attributions up to 60x faster.

2. Aggregation and Ensembling

If a single run is noisy, the average of many runs might be stable. The SACX (Stable Attribution Class Explanation) method, developed by Samuel Rönnqvist and others, aggregates attributions across multiple documents and training iterations to filter out noise (Explaining Classes through Stable Word Attributions). Similarly, Prashnna K. Gyawali shows that ensembling feature importance scores across different hyperparameters and epochs significantly boosts the power and stability of feature selection (Ensembling improves stability and power of feature selection for deep learning models).

3. Advanced Sampling and Distribution Compression

The "Kernel Thinning" approach introduced by Hubert Baniecki and Giuseppe Casalicchio replaces random sampling with "distribution compression." By selecting a representative "coreset" of data points rather than relying on i.i.d. sampling, they reduce explanation variance while requiring fewer model evaluations (Efficient and Accurate Explanation Estimation with Distribution Compression).

4. Incorporating Structural Priors

Stability can also be achieved by forcing the model or the explainer to respect known structures. BayLIME, proposed by Xingyu Zhao and Xiaowei Huang, uses Bayesian priors to stabilize the local surrogate models used in LIME, making them more robust to changes in sampling parameters (BayLIME: Bayesian Local Interpretable Model-Agnostic Explanations).

The Fidelity-Stability Trade-off

While stabilization is necessary for trust, it is not free. Farzan Farnia and colleagues mathematically demonstrate a fundamental trade-off: as you increase the stability of a saliency map (for example, through Gaussian smoothing), you often decrease its fidelity to the model's original, local decision logic (On the Trade-Off between Stability and Fidelity of Gaussian-Smoothed Saliency Maps).

The goal of this research direction is not just to make explanations "static," but to ensure that the stability we see reflects the true underlying mechanism of the model, rather than a smoothed-over version of its errors.

Papers in this version

Showing 60 of 285 papers.

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 164 papers
  2. Version 1 · 2026-10-04 · 285 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_explanation_instability_stochastic_expla_5a30a1d8,
  title = {explanation instability (stochastic explanation variance)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/explanation-instability-stochastic-explanation-variance/rdr_5a30a1d81e74435b99c487e2b53968de},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_stabilizing_stochastic_explanations_for__5a30a1d8_v1,
  title = {Stabilizing Stochastic Explanations for Reliable Interpretations},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/stabilizing-stochastic-explanations-for-reliable-interpretations/rdr_5a30a1d81e74435b99c487e2b53968de/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}