Domain adaptation with optimal transport improves EEG sleep stage classifiers

Stanislas Chambon, Mathieu N. Galtier, Alexandre Gramfort

TL;DR

This paper tackles the challenge of accurately classifying sleep stages from EEG data when the model trained on one dataset (source) performs poorly on a new, different dataset (target) due to data discrepancies. They demonstrate that using Optimal Transport-based Domain Adaptation (OTDA) significantly improves classifier performance on the target domain, recovering a substantial portion of performance lost due to this "dataset bias."

What Problem Does This Paper Address?

Training robust machine learning models, especially for complex tasks like sleep stage classification from EEG, often requires large amounts of labeled data. However, in real-world scenarios, data collected from different individuals, equipment, or even at different times can exhibit significant statistical differences – a phenomenon known as "dataset bias" or "domain shift." A model trained on one dataset (the "source") may perform poorly when applied to another dataset (the "target") because the underlying data distributions don't match. This is particularly problematic in medical applications like sleep analysis, where obtaining large, consistently labeled datasets can be difficult and expensive. This paper focuses on addressing this domain shift problem specifically for EEG sleep stage classification.

Methods

The core of the paper's approach lies in Domain Adaptation (DA), a set of techniques designed to bridge the gap between source and target domains. They specifically leverage Optimal Transport Domain Adaptation (OTDA) methods. Here's a breakdown:

  • Optimal Transport (OT): At its heart, OT is about finding the most efficient way to move "mass" from one distribution to another. In this context, it's used to find a mapping that transports features from the source domain to align with the distribution of features in the target domain.
  • Entropic Regularization: Standard OT can be computationally expensive. The authors employ entropic regularization, which makes the optimization problem more tractable and leads to a faster solution using the Sinkhorn-Knopp algorithm. This helps in efficiently finding the optimal "transportation plan" between source and target features.
  • Regularization on Source Labels (LpL1): To further structure the transport, they incorporate a regularization term that encourages source samples with the same label to be transported to similar target samples, and vice-versa for different labels. This is referred to as the LpL1 method.
  • Deep Learning Architecture: The base classifier is a convolutional neural network (CNN) previously shown to be effective for sleep stage classification. The DA methods are applied to the features extracted from the penultimate layer of this CNN, before the final classification layer.
  • General Approach: The workflow involves:
  • Extracting features from the source and target data.
  • Applying OTDA (OT or LpL1) to adapt the source features to the target domain.
  • Retraining the classification layer of the CNN using these adapted source features.
  • Predicting sleep stages on the target data using the retrained classifier.

Datasets

  • Source Domain: MESA dataset, containing EEG and EOG signals from 3 EEG derivations and 2 EOG derivations.
  • Target Domain: MASS-session 3 dataset, used for evaluation. The authors selected derivations that were similar to those in the MESA dataset.

The paper does not introduce a new benchmark dataset but rather uses existing ones to evaluate their domain adaptation approach.

Key Insights and Ideas

  • OTDA is a Powerful Tool for EEG Sleep Stage Classification: The most significant insight is that OTDA methods, particularly the LpL1 variant, can effectively mitigate the performance degradation caused by domain shift in EEG sleep stage classification.
  • Recovering Lost Performance: The study shows that OTDA methods can recover a substantial portion of the performance lost when a model is directly applied to a new domain, highlighting its practical utility.
  • Hyperparameter Optimization is Crucial but Manageable: While hyperparameters like $\epsilon$ and $\eta$ in OTDA methods are important, the paper demonstrates that a "maximum heuristic" (setting hyperparameters based on the maximum feature values) can yield good results without extensive grid search, simplifying the adaptation process.
  • Beyond Simple Alignment: The LpL1 method's ability to leverage source labels during adaptation provides a more structured and effective way to align distributions compared to methods that only consider feature distributions.

Key Results

  • Significant Performance Improvement: OTDA methods (OT and LpL1) outperformed baseline DA methods like Subspace Alignment (SA) and Correlation Alignment (CORAL) by a margin of over 5% in averaged balanced accuracy.
  • Improved Stage Detection: Per-class metrics and confusion matrices showed that OTDA methods led to more diagonal confusion matrices, indicating better detection across all sleep stages, with improved sensitivity, specificity, and precision for most stages.
  • LpL1 Outperforms OT: The LpL1 method, which incorporates label information, generally showed slightly better performance than the standard OT method.
  • Maximum Heuristic Effectiveness: Using the maximum heuristic for hyperparameter tuning for LpL1 resulted in performances close to or exceeding those found through grid search, and significantly improved performance on a per-record basis.

Limitations and Future Work

  • Limitations:
    • The paper acknowledges that while OTDA methods improve performance, they don't fully recover the "optimal performance" that would be achieved if labels were available for the target domain (indicated by the dashed line in Figure 1).
  • The computational cost of OTDA, even with entropic regularization, might still be a consideration for very large datasets or real-time applications, although it's presented as significantly improved over non-regularized OT.
  • Future Work:
    • The authors suggest that their approach is relevant for any predictive model applied to new datasets with distribution shifts, implying its potential applicability beyond sleep stage classification.
  • Further exploration of different feature extraction methods or deep learning architectures in conjunction with OTDA could be beneficial.
  • Investigating more advanced regularization techniques or adaptive hyperparameter selection strategies could further enhance performance and robustness.

Summary updated

Abstract

Low sample size and the absence of labels on certain data limits the performances of predictive algorithms. To overcome this problem, it is sometimes possible to learn a model on a large labeled auxiliary dataset. Yet, this assumes that the two datasets exhibit similar statistical properties which is rarely the case in practice: there is a discrepancy between the large dataset, called the source, and the dataset of interest, called the target. Improving the prediction performance on the target domain by reducing the distribution discrepancy, between the source and the target domains, is known as Domain Adaptation (DA). Presently, Optimal transport DA (OTDA) methods yield state-of-the-art performances on several DA problems. In this paper, we consider the problem of sleep stage classification, and use OTDA to improve the performances of a convolutional neural network. We use features learnt from the electroencephalogram (EEG) and the electrooculogram (EOG) signals. Our results demonstrate that the method significantly improves the network predictions on the target data.
BibTeX
@inproceedings{chambon2018domain,
  author = {Stanislas Chambon and Mathieu N. Galtier and Alexandre Gramfort},
  title = {{Domain adaptation with optimal transport improves EEG sleep stage classifiers}},
  booktitle = {PRNI},
  year = {2018},
  doi = {10.1109/prni.2018.8423957},
}
Figure