Identifying Drivers of Predictive Aleatoric Uncertainty

Pascal Iversen, Simon Witzke, Katharina Baum, Bernhard Y. RenardIJCAI 2025 · DOI

TL;DR

This paper introduces a straightforward method to explain aleatoric uncertainty in regression models, a crucial but often overlooked aspect of AI explainability. By adapting standard neural networks to predict variance and then applying off-the-shelf explainers, they demonstrate a more reliable way to understand why a model is uncertain, outperforming more complex existing methods on various benchmarks.

What Problem Does This Paper Address?

In safety-critical applications, understanding not just what a model predicts but also how confident it is (uncertainty quantification) is paramount for trust and reliability. While point predictions are increasingly explainable, the drivers behind uncertainty estimates themselves remain largely a black box. This lack of transparency hinders our ability to trust AI systems, especially when they signal high uncertainty. Existing methods for explaining uncertainty are often complex, relying on Bayesian neural networks or intricate generative models, limiting their practical adoption. This paper aims to bridge this gap by providing an accessible and effective method to explain aleatoric uncertainty, which arises from inherent noise in the data.

Methods

The core of the proposed method, termed Variance Feature Attribution (VFA), involves two main steps:

  1. Deep Heteroscedastic Regression: Standard regression models typically predict the mean of the target variable. Here, the authors extend these models (specifically, neural networks) to predict both the mean and the variance of a Gaussian distribution. This allows the model to capture input-dependent noise, which is the source of aleatoric uncertainty. They use a two-stage training process: an initial MSE-based training for stability, followed by fine-tuning with Gaussian Negative Log-Likelihood (GNLL) to optimize for both mean and variance. For pre-trained models, they suggest a simple extension by adding a column of randomly initialized weights to the output layer to predict variance.
  2. Post-hoc Explanation of Variance: Once the model can predict variance, the authors treat this variance output as another prediction to be explained. They leverage existing, off-the-shelf explainability methods (like SHAP, Integrated Gradients, LRP) to attribute the predicted variance back to the input features. This allows them to identify which features contribute most to the model's uncertainty for a given prediction.

Datasets

The authors evaluate their approach on a combination of synthetic and real-world datasets, including a newly introduced benchmark:

  • Synthetic Datasets: Used for controlled experiments to understand the method's ability to detect known uncertainty drivers.
  • Tabular Real-World Datasets: UCI Wine Quality, Ailerons, and LSAT academic performance datasets were used. To evaluate uncertainty explanation metrics on these, they were augmented with synthetic noise, creating semi-synthetic settings.
  • MNIST+U Dataset: A novel benchmark created by extending the MNIST dataset. It involves placing two MNIST digits on a canvas, where one digit represents the mean and the other the standard deviation of a target Gaussian distribution, creating a high-dimensional image regression task with controllable uncertainty.

Key Insights and Ideas

  • Uncertainty Explanations Complement Point Predictions: Explaining the variance output provides distinct and valuable information compared to explaining the mean prediction. This is crucial for understanding why a model is uncertain, not just what it predicts.
  • Simplicity is Key: The proposed method is remarkably simple, requiring minimal modifications to existing regression models and leveraging standard XAI tools. This makes it highly accessible and practical for wider adoption.
  • VFA Outperforms Complex Methods: The Variance Feature Attribution (VFA) approach, particularly VFA-SHAP, consistently demonstrates superior performance in identifying uncertainty drivers compared to more complex methods like CLUE and InfoSHAP, especially in challenging scenarios.
  • Robustness Matters: The evaluation of robustness (via Lipschitz continuity) highlights that VFA-SHAP and VFA-IG are generally more robust explainers, a critical property for reliable AI systems.
  • Dataset-Specific Explainer Choice: While VFA methods generally perform well, the best choice of explainer can still be dataset-dependent, emphasizing the need for careful evaluation.

Key Results

  • Synthetic Data: VFA-SHAP accurately identified all five ground-truth noise features driving uncertainty, which were distinct from features influencing the mean.
  • Global Explanations: VFA flavors effectively identified uncertainty drivers for high-uncertainty instances, achieving GRAs close to 1. They also showed superior GMA, indicating their ability to disregard irrelevant features.
  • Local Accuracies (RRA/RMA): On tabular datasets, VFA consistently outperformed InfoSHAP and CLUE, especially in complex noise scenarios. VFA-SHAP was the top performer in most complex settings.
  • Robustness: VFA-SHAP and VFA-IG showed the lowest median Lipschitz estimates across most datasets, indicating higher robustness.
  • Faithfulness: Perturbing the most important uncertainty drivers identified by VFA generally led to a negative change in the correlation between residuals and uncertainties, indicating faithful explanations.
  • Image Data (MNIST+U): VFA-LRP performed best, showing the highest relevance attribution to the variance mask, aligning with expectations for this dataset. VFA-IG performed poorly, assigning similar relevance to mean and variance masks.

Limitations and Future Work

  • Aleatoric Uncertainty Focus: The current work focuses exclusively on aleatoric uncertainty. The authors acknowledge that explaining epistemic uncertainty is equally crucial for safety-critical settings and suggest it as a direction for future work.
  • Synthetic Noise Dependency: The evaluation of accuracy metrics relies on introducing synthetic noise, which may not perfectly capture the complexity of real-world noise patterns.
  • Computational Complexity: As VFA inherits the computational costs of the underlying XAI methods, complexity remains a consideration.
  • Synergies in Explanations: The authors suggest exploring synergies between explaining point predictions and uncertainty predictions, particularly in contexts like explainable active learning.

Summary updated

BibTeX
@inproceedings{iversen2025identifying,
  author = {Pascal Iversen and Simon Witzke and Katharina Baum and Bernhard Y. Renard},
  title = {{Identifying Drivers of Predictive Aleatoric Uncertainty}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.24963/ijcai.2025/607},
}
Figure