Revealing the Invisible With Model and Data Shrinking for Composite-Database Micro-Expression Recognition

Zhaoqiang Xia, Wei Peng, Huai-Qian Khor, Xiaoyi Feng, Guoying Zhao

Abstract

Micro-expression recognition (MER) faces significant challenges in composite-database scenarios due to domain shifts and the subtle nature of facial movements. This paper proposes a "Model and Data Shrinking" strategy to mitigate overfitting to domain-specific noise. By utilizing a shallow Recurrent Convolutional Network (RCN) with parameter-free modules and low-resolution optical flow maps, the authors achieve state-of-the-art performance on the MEGC2019 benchmark. The findings suggest that reducing model complexity and input resolution can paradoxically enhance the extraction of essential micro-expression features while filtering out irrelevant domain-specific artifacts.

Introduction

Micro-expressions are fleeting, involuntary facial movements that reveal genuine emotions. Recognizing them is incredibly difficult because they are subtle and short-lived. While deep learning has mastered "macro-expressions" (normal smiles or frowns), it struggles with MER, especially in composite-database scenarios.

Most MER research happens on small, controlled individual datasets. However, real-world applications require models to work across different cameras, lighting conditions, and ethnicities (domain shift). When you mix these datasets together, traditional deep models like ResNet or DenseNet often fail. They are so powerful that they "learn" the specific noise and characteristics of each dataset (the domain) rather than the universal, subtle muscle movements of the micro-expression itself.

Methodology

The authors propose a strategy of Model and Data Shrinking. Instead of increasing depth, they utilize shallower architectures and lower-resolution data to force the model to focus on the most essential features.

The RCN Backbone and Modules

The core of the approach is a Recurrent Convolutional Network (RCN). The "recurrent" aspect involves unfolding a single layer with shared parameters to perform multiple convolutions, keeping the model shallow and the parameter count low. Input is represented via Optical Flow Maps derived from the onset and apex frames of a video.

To boost representation power without adding learnable parameters—which would risk overfitting—three novel modules were developed:

  • Wide Expansion (RCN-W): Uses dilated convolutions to capture a larger receptive field.
  • Shortcut Connection (RCN-S): Adds residual-style connections within recurrent layers to improve gradient flow.
  • Attention Unit (RCN-A): A parameter-free soft attention mechanism inspired by Class Activation Mapping (CAM) that helps the model focus on relevant facial regions.
The three proposed parameter-free modules: Wide Expansion, Shortcut Connection, and Attention Unit.

Finally, a gradient-based Neural Architecture Search (NAS) was employed to automatically find the optimal combination of these modules into a robust final architecture (RCN-F).

Experimental Setup

The primary benchmark is the MEGC2019 (Micro-Expression Grand Challenge 2019) dataset, a composite dataset merging SMIC, CASME II, and SAMM. Evaluation follows a Leave-One-Subject-Out (LOSO) protocol to ensure the model generalizes to unseen individuals across different data sources.

Key Insights: The "Shrinking" Paradox

A critical finding of this research is that as input resolution increases beyond $100 \times 100$ or model depth increases, performance on composite datasets drops significantly. High resolution introduces high-frequency noise that deep models mistake for emotional cues.

Performance (UAR) vs. Flow-map Resolution for various RCN models, demonstrating that higher resolutions do not necessarily lead to better recognition.

By shrinking learning complexity, the authors effectively created a "bottleneck" that filters out domain-specific noise (like lighting differences between datasets) while letting the micro-expression signal pass through. This proves that "parameter-free" expressiveness is key in data-scarce environments like MER.

Key Results

The proposed RCN models consistently outperformed much larger, standard architectures on the MEGC2019 composite task:

  • Superiority over SOTA: The RCN-A model achieved an Unweighted Average Recall (UAR) of 0.7190, significantly beating ResNet18 (0.5629) and DenseNet121 (0.3414).
  • Robustness: While standard deep models saw their performance decline when moving from individual to composite datasets, the RCN variants remained stable.
  • Efficiency: Due to the "shrunk" nature of the models, training and testing for all 68 subjects took only about 1.1 hours on a single GPU at $60 \times 60$ resolution.

Conclusion

This work demonstrates that for micro-expression recognition, bigger is not always better. By intentionally shrinking both the model and the data, researchers can avoid the pitfalls of domain-specific overfitting. Future work will explore more sophisticated temporal dynamics while maintaining low model complexity and investigating broader NAS search spaces.

Summary updated

BibTeX
@article{xia2021revealing,
  author = {Zhaoqiang Xia and Wei Peng and Huai-Qian Khor and Xiaoyi Feng and Guoying Zhao},
  title = {{Revealing the Invisible With Model and Data Shrinking for Composite-Database Micro-Expression Recognition}},
  journal = {IEEE Transactions on Image Processing},
  year = {2021},
  url = {https://openreview.net/forum?id=yw0xrjCtEi},
}
Figure