Abstract
Clinical research demonstrates the bidirectional associations between affect and sleep: Major depressive disorder is a prevalent clinical disorder, characterized by alterations in affect and sleep. Conversely, sleep disturbances emerge as prodromal manifestations preceding the onset of the disorder. Despite these close links, few studies investigated depressed daytime affect and its bidirectional associations with sad affect at the time of sleep. This lack of evidence can be attributed to methodological constraints, challenging the assessment of affect during sleep. To address this research gap, the present study introduces an innovative method utilizing Machine Learning and Infrared technology for the unobtrusive assessment of affect during sleep. The objectives of the present proof of concept were as follows: First, to introduce a novel approach for measuring affect during sleep. Second, to determine the bidirectional associations of depressed affect at daytime and sad affect during sleep in a healthy sample. Sad affect during sleep was measured using three infrared cameras with adaptive night vision, recording 63 videos during sleep. Daytime depressed affect was measured by survey, data analysis included general linear modeling (GLM). In line with the first objective, sad affect during sleep was extracted from video-based facial expressions using Artificial Intelligence software frameworks open DBM and open Face (AiCure). In line with the second objective, GLM analysis yielded that higher levels of depressed affect at daytime predicted higher levels of sad affect during sleep significantly. The inverse association was not significant. Conclusionally, we present an innovative method for the unobtrusive measurement of affect during sleep. This method may be particularly advantageous for the early detection of depressed mood states in subclinical populations and samples undergoing transient depressed mood states. Clinical and general implications are discussed.
Introduction
The relationship between mood and sleep is often described as a "chicken and egg" problem: poor sleep can lead to depression, and depression frequently disrupts sleep. While these links are well-established, researchers have historically struggled to measure affect—the experience of feeling or emotion—while a person is actually asleep. Traditional methods have relied on retrospective self-reports, which are prone to memory bias, or invasive sensors like facial Electromyography (EMG) that often disrupt the sleep patterns being studied.
This paper addresses the need for an unobtrusive, objective measurement of affect during sleep to better understand the "affective continuity" between our waking lives and our unconscious processing.
Methodology
In this longitudinal proof-of-concept study, researchers monitored a single healthy participant over 21 nights. The technical pipeline combined hardware and deep learning software to "read" emotions without physical contact. Three infrared (IR) cameras with adaptive night vision were positioned around the bed to ensure facial visibility regardless of sleeping posture.
The team utilized OpenFace and Open DBM, AI tools that use Deep Convolutional Neural Networks (DCNNs) to identify facial landmarks and "Action Units" (AUs). Following the Facial Action Coding System (FACS), "sad affect" was defined as the concurrent activity of three specific muscle movements: the inner brow raiser (AU 1), brow lowerer (AU 4), and lip corner depressor (AU 15).
As shown in the descriptive data, the study captured a mean of 8.20 hours of sleep per night. Interestingly, the participant exhibited "sad" facial expressions for an average of 1.34 hours (approximately 94 minutes) per night, proving that these micro-expressions are frequent enough for meaningful analysis.
Results and Discussion
The study tested two primary hypotheses using General Linear Modeling (GLM). The first hypothesis (H1), that pre-sleep mood predicts sleep affect, was supported. Higher levels of depressed affect prior to sleep significantly predicted higher levels of sad affect during sleep ($p = 0.040$).
Quantitatively, the model revealed a "7-minute rule": for every one-unit increase in reported daytime depressed affect (on a 0-10 scale), the participant exhibited an average of 7 additional minutes of sad facial expressions during the night ($B = 7.00$).
Conversely, the second hypothesis (H2)—that sleep affect predicts next-morning mood—was not supported ($p = 0.249$). This lack of correlation supports the "Sleep to Forget, Sleep to Remember" model, suggesting that sleep may serve to process and "decouple" the emotional charge from memories rather than simply carrying the mood forward into the next day.
Conclusion
This study demonstrates that AI-driven video analysis is a viable digital biomarker for clinical health settings. By moving facial emotion recognition from the wake state into the sleep state, researchers can observe how daytime emotional experiences translate into physical expressions during the night. While limited by a small sample size, this method offers a promising early warning system for subclinical populations and a new tool for understanding the complex interplay between our emotions and our rest.
Summary updated