Efficient Interpretation of Deep Learning Models Using Graph Structure and Cooperative Game Theory: Application to ASD Biomarker Discovery

Xiaoxiao Li, Nicha C. Dvornek, Yuan Zhou, Juntang Zhuang, Pamela Ventola, James S. Duncan

Deep learning models are incredibly good at spotting patterns in medical images, but they are notoriously bad at explaining why they made a specific diagnosis. In the context of Autism Spectrum Disorder (ASD), knowing the "why" is just as important as the "what"—clinicians need to identify specific brain biomarkers to understand the disorder and monitor treatment.

A recent paper from researchers at Yale University proposes a clever way to peek inside these models using a combination of cooperative game theory and graph theory.

TL;DR

The researchers developed two efficient methods—C-SVE and H-SVE—to calculate Shapley values (a fair way to assign "credit" to features) for deep learning models. By leveraging the underlying graph structure of brain connectivity, they reduced the exponential complexity of game theory, allowing them to identify biologically plausible ASD biomarkers from fMRI data.

What Problem Does This Paper Address?

Deep learning classifiers can identify ASD from functional MRI (fMRI) scans with high accuracy. However, identifying which brain regions (features) drove that classification is difficult.

Most current methods use "leave-one-out" testing: they see how the prediction changes if you ignore one feature. This fails miserably when features are conditionally dependent. In the brain, regions don't work in isolation; they function in networks. If two regions are highly correlated, removing one might not change the prediction at all because the other provides the same information. This leads to "myopic" results where important biomarkers are overlooked.

Shapley Value Explanation (SVE) from cooperative game theory solves this by considering all possible combinations (coalitions) of features. The catch? For $N$ features, you have to calculate $2^N$ combinations. With 116+ brain regions, this is computationally impossible.

Methods

The authors propose two ways to simplify SVE by assuming that features only "cooperate" if they are connected in a functional brain graph.

  1. Centralized SVE (C-SVE): This method only calculates the marginal contribution of a feature when it collaborates with its immediate 1-step neighbors in a connectivity graph.
  2. Hierarchical SVE (H-SVE): This approach uses "community detection" to cluster brain regions into small, non-overlapping teams. It then calculates the SVE within each community, assuming that regions in different communities don't interact significantly.
  3. Monte Carlo Approximation: For communities or neighborhoods that are still too large to compute exactly, they use a sampling-based approach to estimate the values.

Crucially, these methods are model-agnostic, meaning they can be applied to any "black-box" classifier without needing to retrain it.

Datasets

  • MNIST: Used as a "sanity check." The researchers parcellated digit images into regions of interest (ROIs) to see if the algorithm could identify the "pixels" humans use to recognize numbers (like the "x-cross" in the middle of an '8').
  • ASD Task-fMRI: Data from 82 children with ASD and 48 healthy controls. The fMRI scans were taken during a "biological motion perception task."
  • AAL Atlas: Used to parcellate the brain into 116 distinct regions to define the "players" in the game.

Key Insights and Ideas

  • The Power of Interaction: The paper demonstrates that "myopic" single-feature testing significantly underestimates the importance of features. SVE captures the "synergy" between brain regions that simpler methods miss.
  • Graph-Based Pruning: The "aha moment" is realizing that we don't need to check every possible coalition of brain regions. By using the Pearson correlation of time series to build a graph, we can ignore coalitions of regions that never actually talk to each other.
  • Fooling the Classifier: A good interpretation method should identify the features that, if removed, most confuse the model. The authors showed that their SVE methods were much better at "fooling" the trained models than standard ROI testing.

Key Results

  • Better Interpretation: On the MNIST dataset, C-SVE and H-SVE identified smoother, more human-interpretable importance maps compared to single-ROI methods.
  • Quantitative Impact: When the "most important" 50% of features (as identified by C-SVE) were corrupted, the model's prediction probability dropped by 0.720, compared to a drop of only 0.335 for the standard single-region method.
  • Biological Consistency: The biomarkers identified (such as the Putamen, Thalamus, and Superior Temporal Gyrus) align with existing neurological literature on ASD, specifically regions involved in social communication and language.
  • Functional Decoding: Using Neurosynth, the authors showed that the identified biomarkers were positively correlated with "higher-level" social concepts (mind, referential) and negatively correlated with "lower-level" skills (language, recognition).

Limitations and Future Work

The authors note that the biomarkers discovered for a Random Forest model differed from those found for a Deep CNN. This suggests that the interpretation is sensitive to the specific model's architecture and the type of input (connectivity vs. activation).

Future work includes testing these methods on different brain atlases and developing more sophisticated community clustering methods. They also aim to "disentangle" the interaction scores to better understand exactly how specific pairs of brain regions work together to influence the model's decision.

Summary updated

Abstract

Discovering imaging biomarkers for autism spectrum disorder (ASD) is critical to help explain ASD and predict or monitor treatment outcomes. Toward this end, deep learning classifiers have recently been used for identifying ASD from functional magnetic resonance imaging (fMRI) with higher accuracy than traditional learning strategies. However, a key challenge with deep learning models is understanding just what image features the network is using, which can in turn be used to define the biomarkers. Current methods extract biomarkers, i.e., important features, by looking at how the prediction changes if “ignoring” one feature at a time. However, this can lead to serious errors if the features are conditionally dependent. In this work, we go beyond looking at only individual features by using Shapley value explanation (SVE) from cooperative game theory. Cooperative game theory is advantageous here because it directly considers the interaction between features and can be applied to any machine learning method, making it a novel, more accurate way of determining instance-wise biomarker importance from deep learning models. A barrier to using SVE is its computational complexity: $$2^N$$ given N features. We explicitly reduce the complexity of SVE computation by two approaches based on the underlying graph structure of the input data: (1) only consider the centralized coalition of each feature; (2) a hierarchical pipeline which first clusters features into small communities, then applies SVE in each community. Monte Carlo approximation can be used for large permutation sets. We first validate our methods on the MNIST dataset and compare to human perception. Next, to insure plausibility of our biomarker results, we train a Random Forest (RF) to classify ASD/control subjects from fMRI and compare SVE results to standard RF-based feature importance. Finally, we show initial results on ranked fMRI biomarkers using SVE on a deep learning classifier for the ASD/control dataset.
BibTeX
@inproceedings{li2019efficient,
  author = {Xiaoxiao Li and Nicha C. Dvornek and Yuan Zhou and Juntang Zhuang and Pamela Ventola and James S. Duncan},
  title = {{Efficient Interpretation of Deep Learning Models Using Graph Structure and Cooperative Game Theory: Application to ASD Biomarker Discovery}},
  booktitle = {IPMI},
  year = {2019},
  doi = {10.1007/978-3-030-20351-1\_56},
}
Figure