Abstract: This study aimed to evaluate the use of novel optomyography (OMG) based smart glasses, OCOsense, for the monitoring and recognition of facial expressions. Experiments were conducted on data gathered from 27 young adult participants, who performed facial expressions varying in intensity, duration, and head movement. The facial expressions included smiling, frowning, raising the eyebrows, and squeezing the eyes. The statistical analysis demonstrated that: (i) OCO sensors based on the principles of OMG can capture distinct variations in cheek and brow movements with a high degree of accuracy and specificity; (ii) Head movement does not have a significant impact on how well these facial expressions are detected. The collected data were also used to train a machine learning model to recognise the four facial expressions and when the face enters a neutral state. We evaluated this model in conditions intended to simulate real-world use, including variations in expression intensity, head movement and glasses position relative to the face. The model demonstrated an overall accuracy of 93% (0.90 f1-score)-evaluated using a leave-one-subject-out cross-validation technique.
Introduction
Facial expressions serve as critical indicators of emotional and mental states, offering invaluable data for tracking conditions such as depression or cognitive decline. However, monitoring these expressions "in the wild" presents significant technical challenges. Current solutions generally fall into two categories, both with notable drawbacks. Computer Vision (CV) systems are sensitive to lighting conditions and raise substantial privacy concerns for continuous monitoring. Conversely, Electromyography (EMG) provides high accuracy but requires electrodes to maintain firm, constant contact with the skin, which is impractical for daily eyewear.
This research explores a third path: Optomyography (OMG). By integrating non-contact optical sensors into a standard glasses frame, the researchers aimed to create a system capable of reliably classifying expressions while navigating real-world variables like head movement and shifting glasses positions.
Methodology and System Design
The study centers on the OCOsense glasses, which house six proprietary OCO sensors. Unlike EMG, these sensors measure minute 3D skin displacements (X, Y, and Z) from a distance of 4 to 30 mm. This non-contact approach allows for a comfortable, non-intrusive form factor suitable for long-term wear.
The technical pipeline for expression recognition involved:
- Signal Processing: Removing linear trends to account for sensor drift, calculating vector magnitudes for 3D movement, and applying rolling median filters for smoothing.
- Feature Engineering: Extracting 162 features (including mean, standard deviation, kurtosis, and skewness) from 18 sensor streams.
- Machine Learning: A Random Forest classifier was trained to recognize four primary expressions (smile, frown, eyebrow raise, squeezed eyes) and a neutral state.
- Validation: The team employed a leave-one-subject-out (LOSO) cross-validation technique to ensure the model's ability to generalize to new users.
Experimental Dataset
The researchers collected a custom dataset from 27 healthy young adult participants across three specific tasks:
- Task A: Posed expressions varying in intensity (low vs. high) and duration (1s vs. 3s).
- Task B: High-intensity expressions performed while simultaneously moving the head (tilting and turning).
- Task C: Expressions performed with the glasses in three different physical positions on the nasal bridge (low, medium, high).
The final processed dataset comprised 361,226 instances, providing a robust foundation for training and testing the classifier.
Results and Discussion
The OCOsense system achieved an overall accuracy of 93% (0.90 F1-score). Statistical analysis confirmed that the OMG sensors effectively capture distinct variations in muscle movement associated with specific expressions.
Key Findings:
- Movement Robustness: A critical finding was that head movement (tilting or turning) did not significantly impact sensor accuracy. This suggests the system is robust enough for active daily life.
- Positional Sensitivity: Accuracy for brow-related movements (frowns and eyebrow raises) improved when the glasses were worn higher on the nasal bridge, bringing the sensors closer to the frontalis and corrugator muscles.
- Subtle Expression Detection: The sensors maintained a high accuracy of 92% even for "low-intensity" expressions, which are often difficult for human observers or CV systems to distinguish.
- Physiological Confusion: The most common error was mistaking a "frown" for "squeezed eyes" (10% of cases). This is physiologically expected, as both expressions involve the activation of the corrugator supercilii muscle.
Conclusion
The OCOsense glasses demonstrate that OMG is a viable, non-intrusive alternative to EMG and CV for facial expression recognition. While this study relied on posed expressions, the high accuracy and robustness to head movement suggest significant potential for real-world mental health monitoring. Future work will focus on validating the system with spontaneous expressions in naturalistic environments and expanding the demographic diversity of the participant pool to ensure universal applicability across different skin tones and ages.
Summary updated