TL;DR
This study demonstrates that incorporating respiratory signals, even those derived from ECG (ECG-derived respiration or EDR), significantly improves machine learning-based sleep stage classification when combined with actigraphy (ACT) and heart rate variability (HRV). Long Short-Term Memory (LSTM) deep learning models showed the best performance, particularly in distinguishing wake and REM sleep, paving the way for more accurate and accessible home sleep monitoring.
What Problem Does This Paper Address?
Reliable sleep monitoring is crucial for diagnosing and managing numerous health conditions. Traditional polysomnography (PSG) in sleep labs is the gold standard but is expensive, resource-intensive, and can be disruptive to natural sleep. Wearable sensors offer a promising alternative for unobtrusive, long-term monitoring at home. However, current wearable-based systems, primarily relying on actigraphy (movement), often overestimate sleep due to a lack of movement during brief wakeful periods. While combining ACT with heart rate variability (HRV) has shown improvements, the role of respiratory signals in large-scale, multimodal sleep stage classification using advanced machine learning (ML) and deep learning (DL) techniques has remained underexplored. This paper aims to fill that gap by systematically evaluating the impact of respiratory information on sleep stage classification performance.
Methods
The researchers conducted a large-scale benchmark study comparing various ML and DL algorithms for sleep stage classification. They evaluated several multimodal approaches, combining:
- Actigraphy (ACT): Raw movement data.
- Heart Rate Variability (HRV): Features derived from beat-to-beat intervals.
- Respiration Rate Variability (RRV): Features derived from a traditional respiration belt.
- ECG-Derived Respiration Rate Variability (ED-RRV): Features derived from the ECG signal, offering a less intrusive alternative to a respiration belt.
They employed five classic ML algorithms (SVM, MLP, AdaBoost, Random Forest, XGBoost) and two DL algorithms (LSTM, TCN). The study systematically compared these algorithms across different input modalities and sleep stage granularities (wake/N1/N2/N3/REM, wake/NREM/REM, and wake/sleep). Feature extraction involved both expert-defined features for ML models and raw data for DL models, with a focus on time-series analysis. Hyperparameter tuning was performed using grid search and Bayesian optimization. Performance was evaluated using the Matthews Correlation Coefficient (MCC), with statistical significance assessed using Mann-Whitney-U tests.
Datasets
The study utilized the MESA Sleep dataset, a large, publicly available dataset containing PSG, ACT, ECG, and respiration belt data from over 1,000 participants. This dataset is a valuable resource as it combines gold-standard PSG with wearable sensor data, allowing for robust benchmarking of different approaches. The data was split into training (80%) and testing (20%) sets.
Key Insights and Ideas
- Respiratory signals are crucial: Adding respiratory information (RRV or ED-RRV) to ACT + HRV significantly boosted classification performance, especially for ML models.
- DL models excel, but ML benefits greatly from respiration: While LSTM consistently performed best across modalities, ML models like XGBoost showed a more substantial relative improvement when respiratory data was included.
- EDR is a viable alternative: ECG-derived respiration (EDR) provided comparable results to traditional respiration belts, highlighting its potential for less obtrusive sleep monitoring.
- Wake and REM detection are key beneficiaries: Respiratory information particularly improved the detection of wake and REM sleep epochs, which are often challenging for ACT-only systems.
- Feature importance matters: Respiratory features (RRV) were found to have the highest impact on model predictions, especially for distinguishing sleep stages, while movement-based ACT features were more influential for wake detection.
Key Results
- LSTM achieved the highest performance: The LSTM model consistently outperformed other algorithms across various modalities and sleep stage granularities. For AASM standard classification (wake/N1/N2/N3/REM), LSTM achieved a median MCC of 0.51.
- Significant performance gains with RRV: Incorporating RRV features improved the F1-score by up to 8.2 percentage points for XGBoost and 0.7 percentage points for LSTM in the wake/N1/N2/N3/REM classification.
- ED-RRV shows promise: Using ED-RRV instead of RRV yielded comparable results, with moderate improvements for ML algorithms.
- Wake/Sleep classification improvement: Adding HRV and RRV features significantly improved sleep-wake detection performance, with wake epoch detection rates increasing by up to 5.4%.
- No significant age bias: Unlike some previous studies, this research found no significant impact of age on classification accuracy, though a decrease was observed for patients with a high apnea-hypopnea index.
Limitations and Future Work
- N1 and N3 detection challenges: Classifying the finer NREM stages (N1 and N3) remained difficult, likely due to subtle physiological differences and class imbalance. Future work could explore cost-sensitive learning or oversampling techniques.
- EDR accuracy: While EDR is a promising alternative, its accuracy can be affected by movement artifacts and noise, leading to slightly lower performance compared to direct respiratory measurements. Future research should focus on developing more robust EDR extraction algorithms.
- Real-world vs. lab data: The MESA dataset was collected in a controlled lab environment. Future studies should validate these findings on real-world, home-monitoring data to assess generalizability.
- Model complexity: While DL models like LSTM performed well, exploring more complex architectures (e.g., ensembles, hybrid models) or incorporating prior physiological knowledge (e.g., sleep cycle periodicity) could further enhance performance.
- Broader sensor integration: Investigating other biosignals like electromyography or electrodermal activity, or using accelerometry instead of actigraphy, could provide additional valuable information.
Summary updated