Evaluating Augmentation Approaches for Deep Learning-based Major Depressive Disorder Diagnosis with Raw Electroencephalogram Data *

Charles A. Ellis, Robyn L. Miller, Vince D. CalhounbioRxiv 2023 · DOI

TL;DR

This paper investigates how data augmentation (DA) techniques can improve deep learning models for diagnosing Major Depressive Disorder (MDD) using raw EEG data, a crucial step for clinical translation given the common issue of small datasets. They found that while most DA methods offer some improvement, simply duplicating the training data often yields comparable results, highlighting the importance of a robust baseline for evaluating new augmentation strategies.

What Problem Does This Paper Address?

Deep learning models show great promise for diagnosing neuropsychiatric disorders like MDD using electroencephalogram (EEG) data. However, a major hurdle for their real-world application is the limited size of available EEG datasets. This scarcity makes models prone to overfitting and hinders their generalization. Data augmentation (DA), a technique to artificially expand training datasets, is a potential solution, but its effectiveness for EEG-based neuropsychiatric diagnosis is not well-understood. This study aims to systematically evaluate various EEG DA methods and compare their performance against a carefully chosen baseline to understand their true utility.

Methods

The researchers developed a deep learning model architecture, previously used for MDD classification, consisting of a feature extractor and a classifier. They employed a subject-wise, group shuffle split for cross-validation to ensure robust evaluation.

The core of their methodology involved comparing several approaches:

  • Baseline Models:
    • M1.1 (No DA): A standard model trained on the original dataset.
  • M1.2 & M1.3 (Duplicate/Triple Data): Models trained on datasets where the original data was simply duplicated once or twice. This serves as a crucial baseline to assess if performance gains are due to augmentation itself or just increased data volume.
  • Data Augmentation (DA) Models (M2-M7): Six different DA techniques were applied to double or triple the training set size. These included:
  • Gaussian noise
    • Time reverse
    • Smooth time masking
    • Fourier transform (FT) surrogate
    • Frequency shift
    • Channel dropout
  • Explainability Analyses: To understand how DA affects the model, they used:
  • Spectral Explainability: This involved perturbing specific frequency bands (delta, theta, alpha, beta, gamma) to see how it impacted model accuracy, revealing which bands are most important for predictions.
  • Spatial Explainability: This method involved ablating (zeroing out) individual EEG channels to assess their importance to the model's performance.

Datasets

The study utilized a publicly available resting-state EEG dataset comprising 30 individuals diagnosed with Major Depressive Disorder (MDD) and 28 healthy controls (HC). Each recording was 5-10 minutes long, sampled at 256 Hz, and used a 10-20 format with 64 electrodes. For analysis, 19 specific channels were selected. The data was preprocessed by low-pass filtering, notch filtering, downsampling to 200 Hz, and channel-wise z-scoring. To increase sample size, the data was segmented into 25-second clips with a 2.5-second sliding window, resulting in 2,942 MDD and 2,950 HC samples.

Key Insights and Ideas

  • The "Duplicate Data" Baseline is Crucial: The most significant methodological contribution is the inclusion of a baseline where the training data is simply duplicated. This is vital because many DA methods increase dataset size, and it's essential to know if the observed improvements are due to the augmentation's novelty or just the sheer volume of data.
  • Augmentation Isn't Always Better Than Duplication: Surprisingly, most DA methods did not significantly outperform the simple "duplicate data" baseline. This suggests that for EEG data, the complexity of augmentation might not always translate to superior performance compared to just having more of the original data.
  • Channel Dropout is a Standout: Channel dropout was the only DA method that consistently improved model performance beyond the duplicate data baseline. This indicates that strategically removing information from specific channels can be a powerful way to enhance model robustness and generalization.
  • Frequency Bands and Channel Importance: The explainability analyses revealed that delta, alpha, and beta frequency bands were generally most important for MDD diagnosis. Interestingly, the FT surrogate DA method seemed to increase model robustness to perturbations in the beta band, even if overall performance wasn't significantly boosted. Spatial analysis showed a tendency for models to rely more on specific channels, and channel dropout DA reduced this reliance.

Key Results

  • No Statistically Significant Differences (Mostly): Despite visual differences in performance metrics (ACC, BACC, SENS, SPEC), the study found no statistically significant differences between most DA approaches and the baseline models after FDR correction.
  • Channel Dropout Outperforms: Model M7.2 (tripled dataset with channel dropout) achieved the highest accuracy (90.75±10.14%), outperforming other DA methods and the simple duplication baselines.
  • Frequency Band Importance: Spectral analysis indicated that delta, alpha, and beta bands were consistently important for MDD classification.
  • Spatial Reliance: Spatial analysis suggested that models trained on duplicated data tended to rely more heavily on individual channels. Channel dropout DA, however, reduced this reliance to negligible levels.

Limitations and Future Work

The authors acknowledge that their findings are specific to their dataset and model architecture. They suggest that future research should:

  • Test Combinations of DA: Explore the synergistic effects of combining different DA approaches and parameters.
  • Evaluate on Multiple Datasets: Validate these findings on diverse EEG datasets and with various model architectures to assess generalizability.
  • Develop New EEG DA Methods: Continue the search for novel EEG DA techniques that can demonstrably improve performance beyond simple data duplication.
  • Compare to Duplicate Baseline: Emphasize the importance of comparing new EEG DA methods against a baseline of duplicated training data to accurately gauge their effectiveness.

Summary updated

Abstract

While deep learning methods are increasingly applied in research contexts for neuropsychiatric disorder diagnosis, small dataset size limits their potential for clinical translation. Data augmentation (DA) could address this limitation, but the utility of EEG DA methods remains relatively underexplored in neuropsychiatric disorder diagnosis. In this study, we train a model for major depressive disorder diagnosis. We then evaluate the utility of 6 EEG DA approaches. Importantly, to remove the bias that could be introduced by comparing performance for models trained on larger augmented training sets to models trained on smaller baseline sets, we also introduce a new baseline trained on duplicate training data to better. We lastly examine the effects of the DA approaches upon representations learned by the model with a pair of explainability analyses. We find that while most approaches boost model performance, they do not improve model performance beyond that of simply using a duplicate training set without DA. The exception to this is channel dropout augmentation, which does improve model performance. These findings suggest the importance of comparing EEG DA methods to a baseline with a duplicate training set of equal size to the augmented training set. We also found that some DA methods increased model robustness to frequency (Fourier transform surrogates) and channel (channel dropout) perturbation. While our findings on EEG DA efficacy are restricted to our dataset and model, we hope that future studies on deep learning for small EEG datasets and on new EEG DA methods will find our findings helpful.
BibTeX
@misc{ellis2023evaluating,
  author = {Charles A. Ellis and Robyn L. Miller and Vince D. Calhoun},
  title = {{Evaluating Augmentation Approaches for Deep Learning-based Major Depressive Disorder Diagnosis with Raw Electroencephalogram Data <sup>*</sup>}},
  year = {2023},
  doi = {10.1101/2023.12.15.571938},
  howpublished = {bioRxiv},
}
Figure