Optomyography-based sensing of facial expression derived arousal and valence in adults with depression

M. John Broulidakis, Ivana Kiprijanovska, Liberty Severs, Simon Stankoski, Martin Gjoreski, Ifigeneia Mavridou, Hristijan Gjoreski, Sophia Cox, Daisy Bradwell, James Stone, Charles NdukaFrontiers in Psychiatry 2023 · DOI

Abstract

We demonstrate for the first time that it is possible to detect facial expression hypo-reactivity in adults with depression in response to emotional content using glasses-based optomyography sensing. It is hoped these results may encourage the use of optomyography-based sensing to track facial expressions in the real-world, outside of a specialized testing environment.

Introduction

Tracking the trajectory of depression currently relies heavily on episodic, subjective measures like clinical interviews or self-reported questionnaires (e.g., the PHQ-9). While clinicians have long recognized that atypical facial expressivity—such as reduced smiling or frowning—is a reliable biomarker for depressive states, measuring these changes objectively outside of a laboratory setting remains a significant challenge. Traditional methods, including electromyography (EMG) or camera-based Facial Action Coding Systems (FACS), are often impractical for continuous, real-world monitoring due to comfort, power, or privacy concerns. This study explores the use of optomyography (OMG) integrated into wearable smart glasses as a discreet, scalable tool for tracking affective states.

Methods

The researchers utilized optomyography (OMG) sensors integrated into Emteq’s OCOsense™ smart glasses. Unlike EMG, which measures electrical activity, OMG uses non-contact optical sensors to measure minute skin displacements caused by underlying muscle movements, specifically targeting the corrugator supercilii (brow) and zygomaticus major (cheek).

A cohort of 75 young adults (38 with depression, 37 healthy controls) participated in the study. While wearing the glasses, participants viewed 25 short video clips from the AVDOS database designed to elicit positive, negative, or neutral emotional responses. Following each clip, participants provided self-reports on emotional arousal and valence. The resulting sensor data was detrended, smoothed, and normalized to compare physical muscle activation against these subjective emotional states.

Figure 1: Normalized signal amplitude of facial muscle movements (cheek and brow) across neutral, negative, and positive video stimuli.

Key Results

The study successfully demonstrated that optomyography is a viable proxy for traditional emotion-tracking methods. Key findings include:

  • Baseline Emotion Tracking: Across all participants, positive videos triggered significant cheek activation, while negative videos triggered significant brow activation. High self-reported arousal correlated with a significant step-increase in muscle activation ($p < 0.001$).
  • Depressive Hypo-reactivity: Participants with depression exhibited measurable "hypo-reactivity"—a blunting of facial muscle movements—specifically in response to negative and highly arousing stimuli. Depressed participants showed significantly less intense facial muscle activation when viewing negative videos ($t_{(67)}=2.58, p=0.012, r=0.30$) and highly arousing videos ($t_{(62)}=2.4, p=0.019, r=0.29$) compared to controls.
  • Emotional Rigidity: Interestingly, the physical reaction to positive stimuli did not differ significantly between groups, suggesting that depression-related hypo-reactivity may be specific to negative emotional processing.

Discussion and Future Work

These results suggest that depression manifests as a form of "emotional rigidity," where the typical physiological response to high-arousal stimuli is dampened. While the study was limited by a controlled laboratory setting and a lack of depression subtyping, it paves the way for objective, continuous monitoring of depressive symptoms in real-world environments. Future research may focus on the dynamics of expressions—such as micro-expression duration and acceleration—to further refine the predictive power of wearable optomyography.

Summary updated

Abstract

We demonstrate for the first time that it is possible to detect facial expression hypo-reactivity in adults with depression in response to emotional content using glasses-based optomyography sensing. It is hoped these results may encourage the use of optomyography-based sensing to track facial expressions in the real-world, outside of a specialized testing environment.
BibTeX
@inproceedings{broulidakis2023optomyography,
  author = {M. John Broulidakis and Ivana Kiprijanovska and Liberty Severs and Simon Stankoski and Martin Gjoreski and Ifigeneia Mavridou and Hristijan Gjoreski and Sophia Cox and Daisy Bradwell and James Stone and Charles Nduka},
  title = {{Optomyography-based sensing of facial expression derived arousal and valence in adults with depression}},
  booktitle = {Frontiers in Psychiatry},
  year = {2023},
  doi = {10.3389/fpsyt.2023.1232433},
}
Figure