An Approach for Estimating Explanation Uncertainty in fMRI dFNC Classification

Charles A. Ellis, Robyn L. Miller, Vince D. Calhoun

TL;DR

This paper introduces a novel method to quantify the uncertainty of "explainability" in brain imaging AI models. By combining Monte Carlo Batch Normalization (MCBN) with Layer-wise Relevance Propagation (LRP), they can estimate how reliable an AI's explanation for a brain scan is, moving beyond single-point estimates to a distribution of possibilities. This is crucial for building trust in AI tools for clinical applications like diagnosing schizophrenia.

What Problem Does This Paper Address?

Neuroimaging studies increasingly use AI models to classify conditions like schizophrenia based on brain activity. To understand why a model makes a certain prediction, researchers employ "explainability" techniques, often gradient-based methods like Layer-wise Relevance Propagation (LRP). However, these methods typically provide a single, definitive explanation (a "point estimate"). This leaves a critical gap: we don't know how confident we should be in that explanation. Is it a robust finding, or could it easily change with slight variations in the data or model? This lack of uncertainty estimation hinders the clinical adoption of these powerful AI tools, as clinicians need to trust not only the prediction but also the reasoning behind it.

Methods

The core innovation lies in combining two techniques:

  • Monte Carlo Batch Normalization (MCBN): This is a method to introduce Bayesian-like uncertainty estimation into standard neural networks. Instead of a single pass through the network, MCBN involves repeatedly processing the data with slightly different batch normalization parameters (derived from iterative updates during training). This generates a distribution of model outputs and, crucially, a distribution of explanations. This is a novel application of MCBN in neuroimaging.
  • Layer-wise Relevance Propagation (LRP): This is a gradient-based explainability technique that assigns a "relevance" score to each input feature (in this case, connections between brain regions) indicating its contribution to the model's prediction.

By applying LRP after each iteration of the MCBN process, the researchers obtain a distribution of LRP explanations for each data sample. They then analyze this distribution to understand the uncertainty associated with the explanations.

Datasets

The study utilizes resting-state functional magnetic resonance imaging (rs-fMRI) data from the Functional Imaging Biomedical Informatics Research Network (FBIRN). This dataset includes 151 individuals diagnosed with schizophrenia (SZs) and 160 healthy controls (HCs). The data was preprocessed to extract 53 independent components (ICs) representing different brain networks, and dynamic functional network connectivity (dFNC) features were calculated across 124 time points for each subject.

Key Insights and Ideas

  • Beyond Point Estimates: The paper convincingly argues that single-point explanations are insufficient for clinical trust. Understanding the variability of an explanation is as important as the explanation itself.
  • MCBN as an Uncertainty Engine: MCBN proves to be a viable and effective method for generating uncertainty estimates in neuroimaging models without negatively impacting model gradients, a common issue with other uncertainty methods like Monte Carlo Dropout.
  • Explanations Differ Between Groups: The study found significant differences in the distribution of LRP relevance between individuals with schizophrenia and healthy controls, suggesting that the AI model learns distinct patterns of brain connectivity for each group.
  • Traditional LRP Can Be Misleading: A key finding is that standard LRP explanations (without MCBN) often diverge from the central tendency of the MCBN-derived explanation distribution. This implies that a single LRP explanation might not always accurately reflect the typical explanation for a given subject or condition.

Key Results

  • Model Performance: The 1D-CNN model achieved above-chance classification performance (mean BACC > 75%) for distinguishing SZs from HCs. MCBN slightly improved performance and helped balance sensitivity and specificity.
  • Inter-Group Differences: Significant differences in relevance were found in most network connectivity pairs between SZs and HCs after MCBN analysis. For example, the interaction between the thalamus (SCN) and cognitive control network (CCN) was more important for HCs, while the interaction of the postcentral gyrus (SMN) with the visual network (VSN) was more important for SZs.
  • Explanation Variability: The study found that traditional LRP relevance differed significantly from the median MCBN-based relevance for a notable percentage of subjects (12-18% of SZs and HCs), particularly for certain network pairs. This highlights the potential for single LRP values to be off-median.

Limitations and Future Work

  • Summation Obscures Detail: The current analysis sums relevance across all time points, which might obscure finer temporal dynamics within the MCBN LRP values.
  • Exploring Other Methods: The authors suggest future work should explore the effects of MCBN on other gradient-based explainability methods and apply the approach to different types of neuroimaging data.
  • Clinical Validation: While the study demonstrates the technical feasibility and potential insights, further validation in real-world clinical settings is implied for full adoption.
  • Broader Applications: The authors aim to further develop this approach for other explainability methods and expect it to enhance the reliability of AI in neuroimaging.

Summary updated

Abstract

Abstract In recent years, many neuroimaging studies have begun to integrate gradient-based explainability methods to provide insight into key features. However, existing explainability approaches typically generate a point estimate of importance and do not provide insight into the degree of uncertainty associated with explanations. In this study, we present a novel approach for estimating explanation uncertainty for convolutional neural networks (CNN) trained on neuroimaging data. We train a CNN for classification of individuals with schizophrenia (SZs) and controls (HCs) using resting state functional magnetic resonance imaging (rs-fMRI) dynamic functional network connectivity (dFNC) data. We apply Monte Carlo batch normalization (MCBN) and generate an explanation following each iteration using layer-wise relevance propagation (LRP). We then examine whether the resulting distribution of explanations differs between SZs and HCs and examine the relationship between MCBN-based LRP explanations and regular LRP explanations. We find a number of significant differences in LRP relevance for SZs and HCs and find that traditional LRP values frequently diverge from the MCBN relevance distribution. This study provides a novel approach for obtaining insight into the level of uncertainty associated with gradient-based explanations in neuroimaging and represents a significant step towards increasing reliability of explainable deep learning methods within a clinical setting.
BibTeX
@inproceedings{ellis2022approach,
  author = {Charles A. Ellis and Robyn L. Miller and Vince D. Calhoun},
  title = {{An Approach for Estimating Explanation Uncertainty in fMRI dFNC Classification}},
  booktitle = {BIBE},
  year = {2022},
  doi = {10.1109/bibe55377.2022.00067},
}

Loading related content...

Figure