SlAction: Non-intrusive, Lightweight Obstructive Sleep Apnea Detection using Infrared Video

You Rim Choi, Gyeongseon Eo, Wonhyuck Youn, Hyojin Lee, Haemin Jang, Dongyoon Kim, Hyunwoo Shin, Hyung-Sin KimarXiv 2023 · DOI · arXiv

Introduction

Obstructive Sleep Apnea (OSA) affects nearly a billion people worldwide, yet it remains vastly underdiagnosed. The current "gold standard" for diagnosis is Polysomnography (PSG), which requires an overnight hospital stay tethered to a dozen sensors. This is not only expensive and uncomfortable but often leads to the "first-night effect," where the unfamiliar environment produces inaccurate sleep data.

While remote monitoring via wearables or IoT sensors exists, these methods often struggle with accuracy or require physical contact. Video-based detection is a promising alternative, but it faces significant hurdles: sleep is mostly "static," blankets obscure chest movements, and sending raw bedroom video to the cloud raises massive privacy concerns. SlAction addresses these challenges with a lightweight, non-intrusive system that detects OSA by analyzing infrared video for "respiratory arousals" rather than breathing itself.

Methodology: A Novel Problem Conversion

The authors introduce a novel "problem conversion" strategy. Instead of trying to detect the subtle, often-occluded breathing patterns of apnea directly, they focus on Respiratory Arousal (RA). These are brief awakenings or movements that occur immediately following an apnea event. As shown in the figure below, the motion intensity during an actual apnea or hypopnea event is minimal, making it difficult to detect under blankets. In contrast, arousals (RA and SA) produce significantly higher motion signatures that are easier for a computer vision system to identify.

Motion intensity (LDPs) across different sleep events, highlighting that arousals (RA, SA) produce significantly more detectable motion than apnea or hypopnea events.

The SlAction pipeline consists of three main stages:

  1. Input Data Preprocessor: The system takes 60-second clips of infrared video at 2.5 FPS. It uses frame differencing to create "difference images," which highlight motion and filter out static background noise.
  2. RA Detector (MoViNet): They utilize MoViNet-A0, a lightweight deep neural network designed for mobile devices. The model performs binary classification to determine if a clip contains an RA event.
  3. AHI Estimator: The system counts detected RA events and uses a Huber Regressor to estimate the Apnea-Hypopnea Index (AHI). If the estimated AHI is $\ge 15$, the patient is classified as having OSA.

Key Insights: Motion as a Proxy

The core innovation of SlAction is the realization that while breathing is hard to see under a duvet, the struggle to breathe (the arousal) creates distinct motion signatures. Furthermore, the researchers found that RA events have a specific "build-up" in the 15 seconds preceding the movement. By using 60-second windows, the model captures this context, allowing it to distinguish respiratory arousals from spontaneous movements.

Temporal motion patterns showing the distinct increase in activity during respiratory (RA) and spontaneous (SA) arousals compared to the pre-arousal period.

By optimizing the model for the NVIDIA Jetson Nano, the authors ensure that video never leaves the device, maintaining patient privacy. Only the calculated AHI score is shared with medical professionals.

Datasets and Results

The study utilized a massive dataset of 5,098 hours of video from 729 patients across three different hospitals. All videos were synchronized with professional PSG data annotated by experts.

  • High Accuracy: SlAction achieved an average F1 score of 87.6% for OSA detection.
  • Strong Correlation: The estimated AHI showed a high Spearman correlation (up to 0.834) with ground-truth PSG measurements.
  • Real-Time Performance: On a Jetson Nano, the system processes a 60-second clip in just ~3.26 seconds, roughly 19x faster than previous state-of-the-art methods.
  • Generalizability: While performance varied slightly by hospital, "re-fitting" the estimator with a small amount of site-specific data restored accuracy to over 83%.

Limitations and Future Work

The current study is limited by single-night data; since OSA severity can fluctuate, multi-night home monitoring would likely improve reliability. Additionally, while RA detection is robust, extremely heavy blankets might still dampen motion signals. Future work will focus on testing SlAction in diverse home environments to further prove its utility in daily life.

Summary updated

Abstract

Obstructive sleep apnea (OSA) is a prevalent sleep disorder affecting approximately one billion people world-wide. The current gold standard for diagnosing OSA, Polysomnography (PSG), involves an overnight hospital stay with multiple attached sensors, leading to potential inaccuracies due to the first-night effect. To address this, we present SlAction, a non-intrusive OSA detection system for daily sleep environments using infrared videos. Recognizing that sleep videos exhibit minimal motion, this work investigates the fundamental question: "Are respiratory events adequately reflected in human motions during sleep?" Analyzing the largest sleep video dataset of 5,098 hours, we establish correlations between OSA events and human motions during sleep. Our approach uses a low frame rate (2.5 FPS), a large size (60 seconds) and step (30 seconds) for sliding window analysis to capture slow and long-term motions related to OSA. Furthermore, we utilize a lightweight deep neural network for resource-constrained devices, ensuring all video streams are processed locally without compromising privacy. Evaluations show that SlAction achieves an average F1 score of 87.6% in detecting OSA across various environments. Implementing SlAction on NVIDIA Jetson Nano enables real-time inference (~3 seconds for a 60-second video clip), highlighting its potential for early detection and personalized treatment of OSA.
BibTeX
@misc{choi2023slaction,
  author = {You Rim Choi and Gyeongseon Eo and Wonhyuck Youn and Hyojin Lee and Haemin Jang and Dongyoon Kim and Hyunwoo Shin and Hyung-Sin Kim},
  title = {{SlAction: Non-intrusive, Lightweight Obstructive Sleep Apnea Detection using Infrared Video}},
  year = {2023},
  eprint = {2309.02713},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2309.02713},
}
Figure