InsightSleepNet: the interpretable and uncertainty-aware deep learning network for sleep staging using continuous Photoplethysmography

Borum Nam, Beomjun Bark, Jeyeon Lee, In Young KimBmc Medical Informatics Decis. mak 2024 · DOI

TL;DR

This paper introduces InsightSleepNet, a novel deep learning model for automatic sleep staging using only photoplethysmography (PPG) signals from wearable devices. It addresses the limitations of traditional polysomnography by offering an interpretable and uncertainty-aware approach, improving clinical decision-making and patient monitoring.

What Problem Does This Paper Address?

Accurate sleep staging is crucial for diagnosing sleep disorders and assessing sleep quality. Traditional methods, like polysomnography (PSG), are cumbersome, expensive, and require overnight stays, leading to the "first-night effect" which can skew results. While wearable devices offer a more convenient alternative, existing sleep staging algorithms often lack interpretability and struggle with uncertainty in their predictions, making it difficult for medical professionals to trust and act upon the results. This paper aims to bridge this gap by developing a deep learning model that leverages readily available PPG signals, provides insights into its decision-making process, and quantifies its confidence in predictions.

Methods

The core of this work is the InsightSleepNet architecture, a 4-class sleep staging model (Wake, Light Sleep, Deep Sleep, REM Sleep) built upon continuous PPG data. Key components include:

  • Local Attention Module: Inspired by Temporal Convolutional Networks (TCNs), this module helps the model focus on relevant temporal segments of the PPG signal, providing insights into which parts of the signal influence the prediction for each epoch. This contributes to interpretability.
  • InceptionTime Module: This module is used for effectively compressing features and learning from long time series data, leveraging multiple convolutional operations with different time scales and residual connections.
  • Time-Distributed Dense Layer and TCN: These components are combined to process the temporal features extracted by the InceptionTime module.
  • Energy Score for Uncertainty Estimation: The paper employs an energy score to quantify prediction uncertainty. This method helps identify out-of-distribution (OOD) samples or those where the model has low confidence. By setting an energy score threshold, the model can "reject" uncertain predictions, leading to improved accuracy and reliability. This approach is particularly useful for addressing the overconfidence issue in some deep learning models.

Datasets

The study utilized three publicly available polysomnography (PSG) datasets:

  • MESA (Multi-Ethnic Study of Atherosclerosis): Provided data from 2054 subjects with PPG signals sampled at 256 Hz.
  • CFS (Cleveland Family Study): Included data from 320 subjects with PPG signals sampled at 256 Hz.
  • CAP (Cyclic Alternating Pattern): Contained PPG data from 24 subjects with sleep disorders, sampled at 128 Hz.

All datasets were annotated into six classes based on the Rechtschaffen & Kales (R&K) manual, which were then mapped to the four classes used in this study (Wake, Light Sleep, Deep Sleep, REM Sleep).

Key Insights and Ideas

  • PPG is Sufficient for Sleep Staging: The study demonstrates that continuous PPG signals, when processed by a sophisticated deep learning model like InsightSleepNet, can achieve competitive performance for sleep staging, reducing reliance on more complex PSG signals.
  • Interpretability Enhances Trust: The inclusion of a local attention module provides a mechanism to understand why the model makes a certain prediction, which is crucial for gaining the trust of medical professionals and facilitating collaboration between humans and AI.
  • Uncertainty Quantification is Key for Reliability: Using an energy score to estimate prediction uncertainty allows the model to flag ambiguous cases. Rejecting these uncertain predictions significantly improves overall accuracy and reliability, making the model more practical for clinical use.
  • Energy Score Thresholding Improves Performance: Applying an energy score threshold, even without complex model modifications, leads to noticeable improvements in accuracy, Cohen's kappa, and weighted F1 scores across all datasets.

Key Results

  • Improved Performance with Energy Score Thresholding:
    • On the MESA dataset, accuracy increased from 84.2% to a range of 84.8-86.1% after applying the energy score threshold.
  • On the CFS dataset, accuracy improved from 80.6% to a range of 81.9-85.6%.
  • On the CAP dataset, accuracy rose from 80.6% to a range of 81.4-84.3%.
  • Competitive Performance: InsightSleepNet's performance, especially after thresholding, was comparable to or exceeded that of existing state-of-the-art PPG-based sleep staging models.
  • Sleep Parameter Estimation: The model also demonstrated strong correlations (r > 0.8) for estimating sleep parameters like Total Sleep Time (TST) and REM sleep fraction, with further improvements observed after applying the energy score threshold.
  • Interpretability Insights: Analysis of the local attention module showed that high attention scores often corresponded to peaks in the PPG signal, suggesting the model leverages physiological characteristics like heart rate.

Limitations and Future Work

  • Deep Sleep Classification: The paper acknowledges that performance for deep sleep stages is not as high as other stages, likely due to class imbalance. Future work will focus on techniques to address this imbalance and improve sensitivity for deep sleep.
  • 5-Class Sleep Staging: Achieving satisfactory performance for 5-class sleep staging (including N1 and N2 stages separately) using only PPG remains a challenge. The authors plan to develop more detailed models to capture cardiovascular characteristics specific to each stage.
  • Optimal Threshold Determination: While energy score thresholding is effective, determining the optimal threshold currently relies on empirical methods. Future research aims to develop a more theoretically grounded approach for threshold selection.
  • Real-time Application: The current model is designed for offline analysis. Adapting it for real-time application on wearable devices is a potential future direction.
  • Quantifying Interpretability: Further research into quantifying the interpretability of the model could offer even deeper insights.

Summary updated

BibTeX
@article{nam2024insightsleepnet,
  author = {Borum Nam and Beomjun Bark and Jeyeon Lee and In Young Kim},
  title = {{InsightSleepNet: the interpretable and uncertainty-aware deep learning network for sleep staging using continuous Photoplethysmography}},
  journal = {Bmc Medical Informatics Decis. mak},
  year = {2024},
  doi = {10.1186/s12911-024-02437-y},
}
Figure