TL;DR
This paper explores using smartphone sensors for passive sleep monitoring in a diverse, real-world population. They developed machine learning models that achieve good accuracy in detecting sleep times, demonstrating that mobile phones can be a viable tool for scalable sleep tracking outside of controlled lab settings. However, data quality issues like missing sensor data and misreported sleep times present significant challenges that limit accuracy.
What Problem Does This Paper Address?
Accurately monitoring sleep is crucial for understanding health and well-being, but traditional methods like polysomnography are expensive and time-consuming, while self-reported sleep diaries are prone to bias. Recent advancements have explored using mobile phone sensors for passive sleep monitoring. However, this research has largely been confined to small, homogeneous groups (like college students) in controlled environments, raising questions about its generalizability to the broader population in everyday use. This paper aims to bridge that gap by investigating how well mobile phone-based sleep monitoring can scale to a diverse population in their natural environments, identifying both the opportunities and obstacles.
Methods
The researchers employed a two-stage approach for sleep detection. First, they used random forests, an ensemble learning method, to estimate the probability of a 10-minute segment of sensor data corresponding to sleep or wakefulness. This involved extracting 22 distinct features from various phone sensors (activity, light, sound, battery, GPS, WiFi, communication events, and time of day). To account for the temporal nature of sleep, they then used a Hidden Markov Model (HMM) to infer the most likely sequence of sleep/wake states, incorporating transition probabilities between states. They trained models in two ways: global models (trained on data from all participants and tested on a subset) and personal models (trained and tested on data from the same participant).
Datasets
The study utilized data collected from 208 participants across the United States over a 6-week period. The data comprised:
- Mobile phone sensor data: Collected using the Purple Robot app, including activity, light, sound, screen state, battery level, GPS location, WiFi connection, and communication events (calls, SMS).
- Ecological Momentary Assessment (EMA) data: Daily self-reported sleep start and end times.
This dataset represents a novel benchmark for passive sleep monitoring in a large, diverse, real-world population, differing from previous studies that often used smaller, more homogenous groups or controlled settings.
Key Insights and Ideas
- Real-world sleep monitoring is feasible: The study demonstrates that mobile phones, without specific user instructions, can provide adequate sleep monitoring in typical daily use cases.
- Generalizability is achievable, but challenging: The developed methods show promise in generalizing to a broader population than previously studied, but data artifacts in uncontrolled settings pose significant hurdles.
- Time of day is a surprisingly strong predictor: While sensor data is valuable, the time of day alone provides a substantial portion of the predictive power for sleep state.
- Data quality is paramount: Missing sensor data and misreported sleep times are major contributors to classification errors, highlighting the need for robust data cleaning and imputation strategies.
- Personalized models offer a slight edge: While global models are useful, personal models (trained on individual data) generally perform better, suggesting that individual sleep patterns, while somewhat predictable by time, have unique characteristics.
Key Results
- Overall accuracy: Using all sensor features, the average accuracy for classifying 10-minute segments as sleep was 88.8%. Adding time of day increased this to 88.8%. Time of day alone achieved 86.9% accuracy.
- Individual variability: Accuracy varied significantly across participants, ranging from 65.1% to 97.3%.
- Impact of data quality correction: After correcting for missing sensor data and misreports, the average accuracy increased to 91.8%.
- Sleep time estimation: The median absolute deviation (MAD) for sleep start time detection was 38 minutes, and for sleep end time was 36 minutes.
- Sleep duration estimation: The average MAD for predicted sleep duration was 55 minutes.
- Personal vs. Global Models: Personal models performed better for 80.2% of participants, with a high correlation (r=.685) between personal and global model accuracies.
- Missing data impact: A weak but significant inverse relationship was found between classification accuracy and the proportion of missing sensor or EMA data. Missing data during sleep was almost twice as prevalent as during awake states.
- Misreport analysis: An anomaly in reported sleep start times between 12 pm and 3 pm, associated with unusually long sleep durations, was identified as likely due to "am/pm" confusion.
Limitations and Future Work
- Self-reported sleep times are imperfect: The study acknowledges that the ground truth (self-reported sleep times) may not be entirely accurate, potentially affecting the calculated accuracy of the algorithms.
- HMM assumptions: The HMM parameters were set assuming one sleep-to-awake and one awake-to-sleep transition per 24 hours, which may not hold for individuals with fragmented sleep, sleep disorders, or those who take naps.
- Lack of sleep disorder information: The study did not assess for sleep disorders, which could significantly influence sleep patterns and phone interaction.
- Participant demographics: The sample was not perfectly representative of the general US population (e.g., higher proportion of females, requirement for WiFi).
- Phone changes: Some participants changed phones during the study, requiring data management.
Future research should focus on:
- Understanding individual behavior in natural settings to develop more robust sleep monitoring tools.
- Addressing data artifacts and improving data quality, especially for uncontrolled environments.
- Developing methods that can better handle fragmented sleep, naps, and potential sleep disorders.
Summary updated