Automatic Sleep Staging: Recent Development, Challenges, and Future Directions

Huy Phan, Kaare B. MikkelsenarXiv 2021 · DOI · OpenReview

TL;DR

This review paper surveys the rapidly evolving field of automatic sleep staging using EEG signals, highlighting the significant progress made with deep learning. While current methods achieve expert-level performance on healthy subjects, the authors emphasize that widespread clinical adoption is hindered by challenges like data heterogeneity, interpretability, and the need for robust algorithms for diverse patient populations.

What Problem Does This Paper Address?

Sleep staging, the process of classifying sleep into different stages (e.g., wake, N1, N2, N3, REM), is crucial for diagnosing sleep disorders and understanding sleep's role in overall health. Traditionally, this is a manual, time-consuming, and subjective process performed by human experts on polysomnography (PSG) data. This manual approach is a bottleneck for large-scale studies and home-based monitoring. The paper addresses the challenge of automating this process to make sleep assessment more accessible, cost-effective, and scalable, while also exploring its potential for novel clinical applications.

Methods

The paper focuses on deep learning approaches for automatic sleep staging. It discusses the evolution from traditional machine learning to deep neural networks, including:

  • Early Deep Learning: Simple networks (DNN, CNN, RNN) applied to short epoch contexts.
  • Sequence-to-Sequence Framework: A dominant paradigm where an "epoch encoder" extracts features from individual epochs, and a "sequence encoder" models temporal dependencies across multiple epochs. This framework allows for end-to-end training and better captures the context-dependent nature of sleep staging.
  • Hybrid Networks: Combining CNNs for feature extraction with RNNs (like LSTMs) for sequential modeling.
  • Emerging Architectures: Mention of Graph Convolutional Networks (GCNs) and Transformers for epoch encoding, and self-attention mechanisms for sequence encoding.

The review emphasizes the shift towards models that can capture long-range dependencies, mimicking how human experts consider a broader context around an epoch.

Datasets

The paper references several public sleep databases commonly used for training and evaluating automatic sleep staging systems. These include:

  • PhysioNet
  • National Sleep Research Resource (NSRR)
  • EDF-20
  • MASS
  • SHHS
  • DOD-H
  • DOD-O
  • CHAT
  • ISRUС
  • CAP

The authors note that performance can vary significantly across these datasets, particularly for those with sleep pathologies.

Key Insights and Ideas

  • Deep learning has revolutionized sleep staging: The field has moved from conventional methods to deep learning, achieving performance comparable to human experts on healthy individuals.
  • Long-term temporal context is crucial: Mimicking human scoring, models that consider a sequence of epochs (sequence-to-sequence) significantly outperform those that only look at isolated epochs.
  • Clinical adoption is lagging: Despite algorithmic advancements, automatic sleep staging is not yet widely adopted in clinical settings due to several practical and technical barriers.
  • Wearable EEG opens new frontiers: The rise of wearable devices for sleep monitoring presents opportunities for longitudinal studies but also introduces challenges like lower signal quality and smaller datasets.
  • Interpretability is key for trust: For AI to be trusted by clinicians, its decision-making process needs to be understandable, especially given the inherent ambiguity in sleep staging.
  • Data heterogeneity is a major hurdle: Sleep data from different sources (institutions, devices, patient populations) exhibit distributional shifts that degrade model performance.

Key Results

  • Expert-level performance on healthy subjects: State-of-the-art deep learning models have achieved Cohen's kappa scores of ≥ 0.81 on datasets with a majority of healthy subjects, which is considered "almost perfect" agreement.
  • Substandard performance on pathological sleep: Performance remains significantly lower on datasets associated with sleep pathologies.
  • No single "best" architecture: The paper notes that different network architectures within the sequence-to-sequence framework often yield similar "almost perfect" consensus levels on healthy data, suggesting diminishing returns for accuracy improvements alone.

Limitations and Future Work

Limitations:

  • Clinical adoption: Current systems are not robust enough for widespread clinical use, especially for patients with sleep disorders.
  • Data heterogeneity: Models trained on one dataset often perform poorly on data from different sources or patient cohorts.
  • Low signal quality: Wearable devices often produce lower quality signals, impacting performance.
  • Small datasets: Training deep learning models for new devices or specific sleep disorders often involves limited data.
  • Interpretability and trust: The "black-box" nature of deep learning models hinders clinical trust and adoption.
  • Subjectivity in scoring: Manual scoring itself is subjective, leading to noisy labels that are transferred to trained models.

Future Work:

  • Robustness to sleep disorders: Developing algorithms that perform reliably on diverse patient populations.
  • Longitudinal monitoring: Leveraging data from wearable devices for long-term sleep tracking and personalized modeling.
  • Change detection: Identifying sudden shifts in sleep patterns.
  • Subject classification: Developing biomarkers from sleep data.
  • Privacy preservation: Exploring federated learning and on-device model personalization.
  • Compact models: Designing efficient models for edge devices.
  • Explainable AI (XAI): Developing methods to understand and interpret model decisions.
  • Handling noisy labels: Robust training strategies for subjective or imperfect annotations.
  • Scorer personalization: Adapting models to the scoring style of individual clinicians.
  • Domain adaptation: Developing techniques to bridge the gap between different data sources and conditions.
  • Heterogeneity handling: Methods to integrate data from various channels and modalities.

Summary updated

Abstract

Modern deep learning holds a great potential to transform clinical practice on human sleep. Teaching a machine to carry out routine tasks would be a tremendous reduction in workload for clinicians. Sleep staging, a fundamental step in sleep practice, is a suitable task for this and will be the focus in this article. Recently, automatic sleep staging systems have been trained to mimic manual scoring, leading to similar performance to human sleep experts, at least on scoring of healthy subjects. Despite tremendous progress, we have not seen automatic sleep scoring adopted widely in clinical environments. This review aims to give a shared view of the authors on the most recent state-of-the-art development in automatic sleep staging, the challenges that still need to be addressed, and the future directions for automatic sleep scoring to achieve clinical value.
BibTeX
@misc{phan2021automatic,
  author = {Huy Phan and Kaare B. Mikkelsen},
  title = {{Automatic Sleep Staging: Recent Development, Challenges, and Future Directions}},
  year = {2021},
  eprint = {2111.08446},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2111.08446},
}
Figure