Electrocardiogram sleep staging on par with expert polysomnography

Adam M. Jones, Laurent Itti, Bhavin R. ShethmedRxiv 2023 · DOI

TL;DR

Researchers have developed a deep neural network that performs five-stage sleep staging using only a single lead of electrocardiogram (ECG) data. Achieving a Cohen’s kappa of 0.725, the model performs on par with expert human scorers using full polysomnography, potentially moving high-quality sleep studies out of the clinic and into the home.

What Problem Does This Paper Address?

The current "gold standard" for sleep staging is Polysomnography (PSG). If you’ve ever had one, you know the drill: you’re hooked up to dozens of sensors (EEG for brain waves, EOG for eye movement, EMG for muscle tension, etc.) and asked to sleep in an unfamiliar clinic while a technician watches. It’s expensive, intrusive, and often results in "unrepresentative" sleep data because, well, it’s hard to sleep with a face full of wires.

On the other end of the spectrum, we have commercial wearables like smartwatches. While convenient, they generally compare poorly to PSG, often oversimplifying sleep stages or failing to generalize across different ages and health conditions. This paper aims for the "sweet spot": a method that is as convenient as a wearable but as accurate as a clinical sleep lab.

Methods

The authors propose a deep neural network architecture designed to process a full night of ECG data simultaneously. The pipeline consists of three main stages:

  1. Feature Extraction: Convolutional layers extract 40 features from 30-second ECG epochs, which are then fused with demographic data (age, sex) and temporal metadata (clock time, relative position in the night).
  2. Temporal Fusion: A modified Temporal Convolutional Network (TCN) allows the model to look "backward and forward" in time. This is crucial because sleep stages are cyclic; knowing what happened 10 minutes ago (or what happens 10 minutes from now) helps classify the current moment.
  3. Classification: Dense layers output the probability for each of the five sleep stages (Wake, N1, N2, N3, and REM).

The Secret Sauce: The authors introduced a novel kappa-correlated loss function. Standard cross-entropy loss often fails in sleep staging because stages are imbalanced (e.g., N1 sleep makes up less than 5% of the night). Their new loss function uses the geometric mean of class-specific kappas, forcing the model to care about the rare stages just as much as the common ones.

Datasets

The study utilized a massive, diverse dataset of 4,000 recordings from the National Sleep Research Resource (NSRR), spanning five major studies (CCSHS, CFS, CHAT, MESA, and WSC).

  • Diversity: Subjects ranged from 5 to 90 years old, carefully selected to match U.S. census distributions.
  • External Validation: To ensure the model wasn't just "memorizing" specific lab equipment quirks, they tested it on a completely independent dataset (MrOS Sleep Study) and found no drop in performance.

Key Insights and Ideas

  • EEG is Not Indispensable: For decades, the assumption was that you need brain waves (EEG) to accurately stage sleep. This paper challenges that "orthodoxy," showing that the autonomic nervous system's signatures in the heart are sufficient surrogates.
  • Context is King: The model's performance plummeted when the order of sleep epochs was shuffled. This proves the network isn't just looking at heart rate; it's learning the structural "grammar" of a night's sleep.
  • Robustness to Noise: The model remained surprisingly accurate even when 50% relative noise was added to the signal, suggesting it could handle the "messy" data typical of home environments.
  • Real-Time Potential: The authors tested a "causal" version of the model (one that doesn't look into the future). While slightly less accurate than the offline version, it still performed at an expert level, opening the door for real-time sleep interventions (like playing sounds to enhance deep sleep).

Key Results

  • Overall Performance: The model achieved a Cohen’s kappa of 0.725 for five-stage staging. In the world of sleep science, a kappa above 0.6 is considered "substantial" agreement; this model effectively matches the level of agreement between two human experts.
  • Stage-Specific Accuracy: The model was strongest at identifying Wake (κ = 0.87) and REM (κ = 0.82). It struggled most with N1 (κ = 0.37), though notably, human scorers also struggle significantly with this transitional stage.
  • Comparison: The model significantly outperformed previous "EEG-less" models, which typically hover around a kappa of 0.5 to 0.6 for five-stage classification.

Limitations and Future Work

Despite the impressive results, the authors note a few hurdles:

  • REM Sensitivity: The model was statistically slightly worse than human PSG for REM sleep. The authors attribute this to the lack of EOG (eye movement) and EMG (muscle) data, which are the primary markers for REM.
  • Data Quality: To achieve these results, the authors had to discard a significant portion of the original NSRR recordings due to poor ECG quality. Future work will need to make the model even more resilient to "loose lead" artifacts common in home use.
  • Clinical Populations: While the dataset was large, more work is needed to see how the model performs on patients with specific cardiac arrhythmias, which might "confuse" an ECG-only staging system.

The authors call this new field "Cardiosomnography," envisioning a future where a simple chest strap replaces the expensive, wire-laden sleep labs of today.

Summary updated

Abstract

Abstract Accurate classification of sleep stages is crucial in sleep medicine and neuroscience research, providing valuable insights for diagnoses and understanding of brain states. The current gold standard for this task is polysomnography (PSG), an expensive and cumbersome process involving numerous electrodes, often performed in an unfamiliar clinic and professionally annotated. Although commercial devices like smartwatches track sleep, their performance compares poorly with PSG. To address this, we present a neural network that achieves gold-standard levels of agreement using a single lead of electrocardiogram (ECG) data (five-stage Cohen’s kappa = 0.725 on subjects 5 to 90 years old). Our method offers an inexpensive, automated, and convenient alternative. Cardiosomnography, or a sleep study conducted with electrocardiography only, could take expert-level sleep studies outside the confines of clinics and laboratories and into realistic settings. This would render higher-quality studies accessible to a broader community, enabling improved sleep research and sleep-related healthcare interventions.
BibTeX
@misc{jones2023electrocardiogram,
  author = {Adam M. Jones and Laurent Itti and Bhavin R. Sheth},
  title = {{Electrocardiogram sleep staging on par with expert polysomnography}},
  year = {2023},
  doi = {10.1101/2023.10.13.23297018},
  howpublished = {medRxiv},
}
Figure