Automatic Human Sleep Stage Scoring Using Deep Neural Networks

Alexander Malafeev, Dmitry Laptev, Stefan Bauer, Ximena Omlin, Aleksandra Wierzbicka, Adam Wichniak, Wojciech Jernajczyk, Robert Riener, Joachim M. Buhmann, Peter AchermannFrontiers in Neuroscience 2018 · DOI

TL;DR

This paper presents deep learning models for automatically scoring human sleep stages from polysomnographic data, aiming to overcome the time-consuming and subjective nature of manual scoring. They found that deep neural networks (DNNs) processing raw EEG, EOG, and EMG data outperformed traditional feature-based methods, and that incorporating temporal information was crucial for accurate sleep stage classification.

What Problem Does This Paper Address?

Manually scoring sleep stages from polysomnography (PSG) recordings is the gold standard in sleep research and clinical practice. This process involves a human expert visually analyzing multiple physiological signals (EEG, EOG, EMG) and classifying consecutive 30-second epochs into different sleep stages (Wake, REM, NREM stages N1-N3). However, this manual scoring is notoriously time-consuming, labor-intensive, and prone to inter-rater variability, meaning different experts might score the same recording differently. This subjectivity and inefficiency hinder large-scale sleep research and the development of real-time sleep monitoring systems. The paper aims to address this by developing automated sleep scoring algorithms that can achieve performance comparable to human experts.

Methods

The researchers explored several machine learning approaches for sleep stage classification:

  • Random Forest (RF) with Feature Engineering: This traditional approach involved extracting a set of 20 hand-crafted features from the PSG signals (e.g., power in different frequency bands, eye movements, muscle tone). These features were then fed into a Random Forest classifier. To account for the temporal nature of sleep, the RF output was further smoothed using a Hidden Markov Model (HMM) or a median filter.
  • Artificial Neural Networks (ANNs) with Feature Engineering: Similar to the RF approach, ANNs were also trained on the engineered features. Specifically, they used Long Short-Term Memory (LSTM) networks, which are well-suited for sequential data and can learn temporal dependencies.
  • Deep Neural Networks (DNNs) with Raw Data: This was the core novelty. Instead of relying on pre-defined features, these models directly processed raw PSG signals (EEG, EOG, EMG). They employed Convolutional Neural Networks (CNNs) to learn hierarchical features from the raw data, followed by LSTM layers to capture temporal dynamics. They explored different network architectures, including CNN-LSTM models and residual CNNs.

A key aspect of their methodology was the consideration of temporal structure. They recognized that sleep scoring isn't just about individual epochs but also about the sequence of stages. They implemented this by using recurrent networks (LSTMs) and by smoothing the outputs of simpler classifiers.

Datasets

The study utilized two datasets:

  1. Healthy Participants Dataset: This dataset comprised 54 whole-night sleep recordings from healthy young males.
  2. Patient Dataset: This dataset included 22 whole-night sleep recordings and 21 Multiple Sleep Latency Tests (MSLT) from patients with narcolepsy and hypersomnia.

The patient dataset was particularly important for evaluating the generalization capabilities of the models to different sleep pathologies and recording structures (e.g., naps in MSLT). They also experimented with training models on a combined dataset of healthy participants and patients to assess the benefit of diverse training data.

Key Insights and Ideas

  • Raw Data is King (for DNNs): Deep neural networks, particularly CNN-LSTMs, processing raw PSG signals significantly outperformed feature-based methods. This suggests that DNNs can learn more effective and nuanced features directly from the data than humans can engineer.
  • Temporal Context is Crucial: Simply classifying individual epochs is insufficient. Incorporating the temporal sequence of sleep stages, either through recurrent networks or post-hoc smoothing, is vital for accurate scoring.
  • Single-Channel EEG Can Be Surprisingly Effective: The study found that neural networks could achieve decent sleep stage classification using only a single EEG channel, which is remarkable given that human experts rely on multiple signals for certain stages (like REM). This highlights the power of deep learning to extract subtle patterns.
  • Patient Data Matters for Generalization: Models trained solely on healthy data showed a performance drop when applied to patient data. Including patient data in the training set improved performance on patient recordings, underscoring the need for diverse training data to build robust, generalizable sleep scoring systems.
  • Stage 1 is Still a Challenge: Similar to human inter-rater reliability, automatically scoring Stage 1 sleep remained difficult for all algorithms, with lower kappa scores compared to other stages.

Key Results

  • High Performance: Most of the implemented algorithms achieved high Cohen's kappa values (around 0.8) when trained and tested on the same type of data (healthy participants). This performance was comparable to human inter-rater agreement.
  • DNNs Outperform: Deep neural networks (CNN-LSTM) working with raw data generally performed better than feature-based methods (RF and feature-based LSTMs).
  • Generalization: While performance dropped on patient data when models were trained only on healthy data, the DNNs still showed reasonable generalization. Training on a mixed dataset improved performance on both healthy and patient data.
  • Channel Importance: Including EOG and EMG channels generally improved performance over using only EEG, especially for distinguishing certain stages. However, the study noted that noisy EMG signals could degrade performance.
  • Stage 1 Kappa: Kappa values for Stage 1 sleep were consistently lower, around 0.4, which the authors noted is comparable to human inter-scorer agreement for this stage.

Limitations and Future Work

  • Expert Scoring Limitations: The study acknowledges that the "ground truth" was based on a single expert's scoring, which might have introduced biases. Future work could benefit from using consensus scoring from multiple experts.
  • Epoch Length Discrepancies: Differences in epoch lengths between datasets (20s vs. 30s) required conversion, which might have introduced minor discrepancies, particularly at transition phases. Ideally, all data would be scored with the same epoch length.
  • Artifact Handling: Movement artifacts were not explicitly scored as a separate class, and their presence could negatively impact performance. Future work could explore incorporating artifact detection and classification.
  • Limited Training Data Diversity: While they used two datasets, the authors suggest that extending training data to include more laboratories, pathologies, and age groups would further improve robustness.
  • Temporal Structure Metrics: The authors suggest that current metrics like Cohen's kappa treat epochs independently and don't fully capture the quality of scoring sleep episodes. Developing new metrics that account for temporal structure and penalize misclassifications at transitions would be beneficial.
  • On-line Detection: The success of these models opens possibilities for on-line sleep detection, potentially for use in mobile devices or driving simulators.

Summary updated

Abstract

The classification of sleep stages is the first and an important step in the quantitative analysis of polysomnographic recordings. Sleep stage scoring relies heavily on visual pattern recognition by a human expert and is time consuming and subjective. Thus, there is a need for automatic classification. In this work we developed machine learning algorithms for sleep classification: random forest (RF) classification based on features and artificial neural networks (ANNs) working both with features and raw data. We tested our methods in healthy subjects and in patients. Most algorithms yielded good results comparable to human interrater agreement. Our study revealed that deep neural networks (DNNs) working with raw data performed better than feature-based methods. We also demonstrated that taking the local temporal structure of sleep into account a priori is important. Our results demonstrate the utility of neural network architectures for the classification of sleep.
BibTeX
@inproceedings{malafeev2018automatic,
  author = {Alexander Malafeev and Dmitry Laptev and Stefan Bauer and Ximena Omlin and Aleksandra Wierzbicka and Adam Wichniak and Wojciech Jernajczyk and Robert Riener and Joachim M. Buhmann and Peter Achermann},
  title = {{Automatic Human Sleep Stage Scoring Using Deep Neural Networks}},
  booktitle = {Frontiers in Neuroscience},
  year = {2018},
  doi = {10.3389/fnins.2018.00781},
}
Figure