A Boost in Revealing Subtle Facial Expressions: A Consolidated Eulerian Framework

Wei Peng, Xiaopeng Hong, Yingyue Xu, Guoying Zhao

Abstract

Facial micro-expressions (MEs) are fleeting, involuntary movements that provide a window into genuine human emotion. However, their subtlety and transience pose significant challenges for automated recognition. This paper presents ME-Booster, a novel consolidated Eulerian framework that unifies motion magnification and time interpolation into a single linear operation. By eliminating the need for intermediate video generation, ME-Booster achieves state-of-the-art accuracy with a 10x speedup over traditional sequential pipelines.

Introduction

Automated Micro-expression Recognition (MER) is hindered by two primary factors: the minute scale of muscle movements and the extremely short duration of the expressions (typically 1/25 to 1/3 of a second). Traditional approaches address these issues through two distinct pre-processing steps: Motion Magnification (MAG) to amplify subtle movements and Time Interpolation (TIM) to increase frame count.

However, treating these as independent modules leads to a "diluted problem" where errors and noise from the magnification stage propagate into the interpolation stage. Furthermore, the sequential nature of these tasks creates a computational bottleneck, requiring the generation and storage of intermediate video data.

Methodology: The ME-Booster Framework

The ME-Booster addresses these limitations by reformulating both MAG and TIM as linear operations within an Eulerian framework. Unlike Lagrangian methods that track specific coordinates, the Eulerian approach analyzes pixel intensity changes at fixed spatial positions.

The researchers derived two primary matrices: a Magnification Matrix ($W^M$) based on first-order Taylor expansion and a recursive motion-capturing algorithm, and an Interpolation Matrix ($W^I$) derived from a graph spectral method. These are combined into a single consolidated parameter matrix ($W = W^M W^I$).

Comparison of the traditional multi-step pipeline versus the consolidated ME-Booster framework.

This consolidation allows for the simultaneous amplification and temporal stretching of input frames through a single matrix multiplication. The resulting framework is training-free, avoids the artifacts associated with intermediate video products, and is suitable for real-time applications.

Experimental Results

The framework was evaluated on the SMIC and CASMEII databases, which include various sensor types such as High-Speed (HS), Visual (VIS), and Near-Infrared (NIR).

Performance and Efficiency

ME-Booster demonstrated significant improvements in both recognition accuracy and processing speed. On the CASMEII database, the model achieved a peak accuracy of 70.85%, surpassing the previous benchmark of 67.21%. On the SMIC-subHS dataset, accuracy reached 87.32%, representing a nearly 7% improvement over existing methods.

Accuracy comparison across different databases and feature descriptors (HiGO-TOP, HOG-TOP, and LBP-TOP), highlighting the consistent gains achieved by the proposed method.

The efficiency gains were even more pronounced. By eliminating intermediate steps, processing time on the CASMEII database was reduced from 420.89 seconds to just 41.61 seconds—a 10x speedup. Similar performance gains were observed across the SMIC database, confirming the framework's robustness across different lighting conditions and sensor types.

Conclusion and Future Work

The ME-Booster framework represents a significant advancement in the pre-processing of micro-expressions. By mathematically consolidating motion magnification and time interpolation, it overcomes the noise propagation and computational overhead of traditional pipelines. While highly effective at revealing subtle expressions, future research will focus on integrating this framework into end-to-end deep learning models and improving robustness against larger "nuisance" head movements.

Summary updated

BibTeX
@inproceedings{peng2019boost,
  author = {Wei Peng and Xiaopeng Hong and Yingyue Xu and Guoying Zhao},
  title = {{A Boost in Revealing Subtle Facial Expressions: A Consolidated Eulerian Framework}},
  booktitle = {FG},
  year = {2019},
  doi = {10.1109/fg.2019.8756541},
}
Figure