Introduction
Micro-expressions (MEs) are the involuntary "tells" of the human face—brief movements lasting less than half a second that reveal genuine emotions. Due to their subtlety and speed, they are notoriously difficult for both humans and standard computer vision models to detect.
Traditionally, high-performing models for Micro-Expression Recognition (MER) have relied on handcrafted "optical flow" algorithms (like TV-L1) to map pixel movement between frames. However, these methods are computationally expensive and often fail to capture the specific nuances required for deep learning. SelfME introduces a novel framework that moves MER toward an end-to-end, self-supervised approach, learning to extract motion features automatically while leveraging the natural symmetry of spontaneous facial movements.
The SelfME Framework
The SelfME framework operates in two distinct stages: a self-supervised motion learning phase followed by a classification phase utilizing a Symmetric Contrastive Vision Transformer (SCViT).
Motion Learning Stage
In this stage, the model is trained to "reconstruct" a target frame (the apex or peak of the expression) by warping a source frame (the onset or start). The system identifies motion-related keypoints and computes local displacements. To ensure the resulting motion is smooth and continuous, the authors employ a B-spline transformation. This maintains diffeomorphism, preventing the warped image from "tearing" or "folding," which is crucial for preserving the integrity of facial structures.
Classification Stage and Symmetric Contrastive Learning
Once the motion features are extracted, they are fed into a Vision Transformer (ViT). The core innovation here is the Symmetric Contrastive (SC) loss. Because spontaneous expressions are generally symmetric, the researchers introduced a mechanism to force the model to learn similar representations for the left and right sides of the face. This helps the model ignore noise—such as asymmetrical lighting or minor head tilts—and focus on the actual emotional muscle movements.
Key Insights and Results
- Symmetry as a Superpower: Enforcing facial symmetry via SC loss provided a larger performance boost than the motion learning itself, acting as a powerful inductive bias for the Transformer.
- Ditching Handcrafted Flow: By learning motion from scratch, the model becomes more flexible, focusing specifically on the eyes, eyebrows, and mouth—the primary regions for micro-expressions.
- State-of-the-Art Performance: SelfME set a new benchmark on the CASME II dataset, achieving an Unweighted F1-score (UF1) of 0.9078 and an Unweighted Average Recall (UAR) of 0.9290. This significantly outperformed previous state-of-the-art methods like FeatRef.
- Ablation Success: Removing the Symmetric Contrastive component caused the UF1 to drop from approximately 90% to 86%, confirming its vital role in the system's accuracy.
Limitations and Future Work
While SelfME represents a significant leap forward, it faces challenges with grayscale datasets (like SAMM), where the lack of color information can destabilize motion extraction. Additionally, the model currently requires manual identification of onset and apex frames.
Future research aims to evolve SelfME into a unified pipeline capable of both spotting (finding the expression in a continuous video stream) and recognition (classifying the emotion) simultaneously, while also improving robustness against varying head poses.
Summary updated