SLEEPNET: Automated Sleep Staging System via Deep Learning

Siddharth Biswal, Joshua Kulas, Haoqi Sun, Balaji Goparaju, M. Brandon Westover, Matt T. Bianchi, Jimeng SunarXiv 2017 · OpenReview

Abstract

Sleep disorders, such as sleep apnea, parasomnias, and hypersomnia, affect 50-70 million adults in the United States. Overnight polysomnography (PSG), including brain monitoring using electroencephalography (EEG), is a central component of the diagnostic evaluation for sleep disorders. While PSG is conventionally performed by trained technologists, the recent rise of powerful neural network learning algorithms combined with large physiological datasets offers the possibility of automation, potentially making expert-level sleep analysis more widely available. We propose SLEEPNET (Sleep EEG neural network), a deployed annotation tool for sleep staging. SLEEPNET uses a deep recurrent neural network trained on the largest sleep physiology database assembled to date, consisting of PSGs from over 10,000 patients from the Massachusetts General Hospital (MGH) Sleep Laboratory. SLEEPNET achieves human-level annotation performance on an independent test set of 1,000 EEGs, with an average accuracy of 85.76% and algorithm-expert inter-rater agreement (IRA) of kappa = 79.46%, comparable to expert-expert IRA.

Introduction

Sleep disorders like apnea and insomnia affect upwards of 70 million adults in the US alone. The "gold standard" for diagnosis is overnight polysomnography (PSG), where a patient is monitored via various sensors, including an EEG to track brain activity. Currently, the primary bottleneck in diagnosis is the manual labor required: a trained technologist must label every 30-second window (epoch) of an 8-hour recording into one of five sleep stages. This process typically takes over an hour per patient and is subject to significant human error and inter-rater variability. While automated tools have been proposed, they often struggle with the "messy" data found in real-world clinical settings due to small training datasets.

Methods and Data

The researchers developed SLEEPNET to address the scale and variety of clinical EEG data. The system was trained on a massive dataset from the Massachusetts General Hospital (MGH) Sleep Laboratory, encompassing over 10,000 patients and 80,000 hours of EEG data (3.2 TB). This dataset is nearly 20 times larger than those used in most prior studies.

The team explored several data representations and architectures, ranging from basic machine learning to deep neural networks.

Classification architectures explored, ranging from basic machine learning to advanced deep learning models.

The best-performing architecture was a 5-layer Long Short-Term Memory (LSTM) network using expert-defined features (96 specific metrics like power ratios and signal complexity). The LSTM's recurrent nature allows it to incorporate temporal context, mimicking how human experts look at the "neighborhood" of data around an epoch to determine its stage.

Results and Clinical Deployment

SLEEPNET achieved human-level performance, with an average accuracy of 85.76% and a Cohen’s Kappa of 79.46%. This level of agreement is comparable to, and often exceeds, the 65-75% agreement rate typically found between human experts.

The SLEEPNET interface showing EEG waveforms, spectrograms, and a comparison between human-annotated and predicted sleep stages.

Beyond accuracy, SLEEPNET offers immense efficiency, scoring an entire 8-hour PSG recording in just 2 to 5 minutes. The system has been deployed at MGH via a web interface, allowing clinicians to review automated labels alongside raw data in real-time.

Discussion and Future Work

A key finding was that "pure" deep learning (raw data + CNN) was outperformed by models using expert-engineered features, highlighting the continued value of domain knowledge in medical AI. Furthermore, model performance scaled consistently with the number of training subjects, suggesting that large-scale datasets are critical for clinical-grade automation.

Future work will focus on improving accuracy for the N1 (light sleep) stage, which remains challenging for both humans and machines. Additionally, the team aims to adapt SLEEPNET for at-home monitoring devices using fewer EEG channels, potentially democratizing access to high-quality sleep diagnostics.

Summary updated

Abstract

Sleep disorders, such as sleep apnea, parasomnias, and hypersomnia, affect 50-70 million adults in the United States (Hillman et al., 2006). Overnight polysomnography (PSG), including brain monitoring using electroencephalography (EEG), is a central component of the diagnostic evaluation for sleep disorders. While PSG is conventionally performed by trained technologists, the recent rise of powerful neural network learning algorithms combined with large physiological datasets offers the possibility of automation, potentially making expert-level sleep analysis more widely available. We propose SLEEPNET (Sleep EEG neural network), a deployed annotation tool for sleep staging. SLEEPNET uses a deep recurrent neural network trained on the largest sleep physiology database assembled to date, consisting of PSGs from over 10,000 patients from the Massachusetts General Hospital (MGH) Sleep Laboratory. SLEEPNET achieves human-level annotation performance on an independent test set of 1,000 EEGs, with an average accuracy of 85.76% and algorithm-expert inter-rater agreement (IRA) of kappa = 79.46%, comparable to expert-expert IRA.
BibTeX
@misc{biswal2017sleepnet,
  author = {Siddharth Biswal and Joshua Kulas and Haoqi Sun and Balaji Goparaju and M. Brandon Westover and Matt T. Bianchi and Jimeng Sun},
  title = {{SLEEPNET: Automated Sleep Staging System via Deep Learning}},
  year = {2017},
  eprint = {1707.08262},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1707.08262},
}
Figure