A Novel Local Ablation Approach for Explaining Multimodal Classifiers

Charles A. Ellis, Darwin A. Carbajal, Rongen Zhang, Mohammad S.e. Sendi, Robyn L. Miller, Vince D. Calhoun, May D. WangbioRxiv 2021 · DOI

TL;DR

This paper introduces a novel "local ablation" technique to understand how different data sources (modalities) contribute to a multimodal deep learning model's decision for each individual sample. Applied to sleep stage classification, it reveals that the model learns meaningful relationships between patient demographics (age, sex, medication) and the importance of different signals, suggesting the model isn't just learning from the raw data but also from these contextual factors.

What Problem Does This Paper Address?

Deep learning models are increasingly using multiple data sources (modalities) like EEG, EOG, and EMG to improve classification accuracy in healthcare. However, these complex models are often "black boxes," making it hard to understand why they make a specific prediction for a given patient. While global explainability methods tell us which features are generally important, they don't explain individual predictions. Local explainability methods aim to do this, but applying them to multimodal data is challenging. Furthermore, understanding how patient-specific factors like age, sex, or medication influence these multimodal models is largely unexplored. This paper addresses the need for local explainability in multimodal settings and investigates how demographic and clinical variables might be implicitly learned by such models.

Methods

The core contribution is a novel local ablation approach. For each individual data sample, the researchers:

  1. Record the model's confidence in its top prediction.
  2. Systematically "ablate" (remove or corrupt) one modality at a time by replacing it with a combination of Gaussian noise and a 40 Hz sinusoid (mimicking line noise).
  3. Re-evaluate the model's confidence in its top prediction with the ablated modality.
  4. Calculate the percentage change in confidence. This change quantifies the importance of the removed modality for that specific sample's classification.

This process is repeated for each modality (EEG, EOG, EMG) and for every sample in the test set. They also perform statistical analyses (Pearson correlation for age, t-tests for sex and medication) to link these local importance scores to demographic and clinical variables.

Datasets

The study uses data from the Sleep-EDF Database (Physionet). This dataset includes polysomnography recordings from 22 subjects (7 male, 15 female), with each subject having two recordings: one after placebo and one after temazepam. The recordings consist of EEG, EOG, and EMG signals sampled at 100 Hz. The data was preprocessed into 30-second samples, with NREM3 and NREM4 stages consolidated.

Key Insights and Ideas

  • Modality Importance Varies by Sleep Stage: EEG is generally the most important modality, but EOG plays a surprisingly crucial role in identifying the NREM1 sleep stage, which is unexpected as EOG is typically associated with Awake and REM.
  • Demographics Matter (Implicitly): The model doesn't just learn from the raw physiological signals; it appears to learn associations between patient demographics (age, sex, medication status) and the importance of different modalities for classification. This suggests the model is picking up on subtle patterns related to these factors.
  • Local Explanations Reveal Nuances: Visualizing local ablation results over time shows how modality importance shifts within a sleep cycle, offering a dynamic view of the classifier's decision-making process.
  • Potential for Robustness: For the "Awake" stage, the classifier showed good performance and relatively low modality importance changes upon ablation. This might indicate the model has developed a robust representation of Awake that relies less on any single modality.

Key Results

  • Classification Performance: The paper reports classification performance metrics (F1, Precision, Recall) for each sleep stage. NREM2 achieved the highest F1 score, while NREM1 had the lowest, likely due to its small sample size.
  • Modality Importance:
    • EEG was the most important modality for most sleep stages.
  • EOG was particularly important for NREM1.
  • For REM, EEG and EOG were both important, with EEG becoming more dominant deeper into REM.
  • Demographic/Clinical Variable Relationships:
    • Medication: Placebo samples generally showed higher EEG importance than temazepam samples (except for NREM2 and REM).
  • Sex: Males tended to have higher EOG importance and lower EMG/EEG importance compared to females.
  • Age: REM EOG activity was strongly correlated with age.
  • Statistical Significance: Many relationships between local ablation results and demographic/clinical variables were statistically significant (p < 0.05, with many reaching p < 0.01 or p < 0.001), supporting the idea that the classifier learned these associations.

Limitations and Future Work

  • Ablation Limitations: The authors acknowledge that ablation can create "out-of-distribution" samples, potentially leading to inaccurate importance estimates. They mitigated this by using noise mimicking line-related artifacts, but other local explainability methods might offer more reliable explanations.
  • Future Directions: The authors suggest comparing their local ablation results with global ablation approaches. They also believe their method can be applied to a wider range of multimodal classification problems and can help guide future biomarker discovery by understanding how demographic and clinical variables influence classifiers.

Summary updated

Abstract

ABSTRACT With the growing use of multimodal data for deep learning classification in healthcare research, more studies have begun to present explainability methods for insight into multimodal classifiers. Among these studies, few have utilized local explainability methods, which could provide (1) insight into the classification of each sample and (2) an opportunity to better understand the effects of demographic and clinical variables within datasets (e.g., medication of subjects in electrophysiology data). To the best of our knowledge, this opportunity has not yet been explored within multimodal classification. We present a novel local ablation approach that shows the importance of each modality to the correct classification of each class and explore the effects of demographic and clinical variables upon the classifier. As a use-case, we train a convolutional neural network for automated sleep staging with electroencephalogram (EEG), electrooculogram (EOG), and electromyogram (EMG) data. We find that EEG is the most important modality across most stages, though EOG is particular important for non-rapid eye movement stage 1. Further, we identify significant relationships between the local explanations and subject age, sex, and state of medication which suggest that the classifier learned specific features associated with these variables across multiple modalities and correctly classified samples. Our novel explainability approach has implications for many fields involving multimodal classification. Moreover, our examination of the degree to which demographic and clinical variables may affect classifiers could provide direction for future studies in automated biomarker discovery.
BibTeX
@misc{ellis2021novel,
  author = {Charles A. Ellis and Darwin A. Carbajal and Rongen Zhang and Mohammad S.E. Sendi and Robyn L. Miller and Vince D. Calhoun and May D. Wang},
  title = {{A Novel Local Ablation Approach for Explaining Multimodal Classifiers}},
  year = {2021},
  doi = {10.1101/2021.06.10.447986},
  howpublished = {bioRxiv},
}
Figure