A Multiview Representation Framework for Micro-Expression Recognition

Tianhuan Huang, Lei Chen, Yuncong Feng, Xianye Ben, Ruixue Xiao, Tianle XueIEEE Access 2019 · DOI

Abstract

Micro-expression recognition faces significant challenges due to the scarcity of labeled data and the subtlety of facial movements. This paper presents a multiview representation framework that leverages transfer learning from macro-expressions (auxiliary domain) to micro-expressions (target domain). By projecting features into a common space and employing sparse dictionary reconstruction alongside multiview feature selection, the proposed method effectively addresses the small sample problem. Experimental results on benchmark databases demonstrate superior performance compared to existing state-of-the-art techniques.

Introduction

Micro-expressions are fleeting, involuntary facial movements that reveal suppressed emotions, typically lasting less than 1/25th of a second. Their detection is vital for fields like clinical psychology and security. However, automated recognition is hindered by the Small Sample Problem—the high cost of labeling makes datasets small—and Feature Uncertainty, as no single descriptor perfectly captures these subtle movements. This research explores whether abundant macro-expression data can serve as a "mentor" to improve micro-expression recognition.

Methodology

The proposed framework utilizes a three-stage pipeline to bridge the gap between macro and micro expressions.

1. Feature Projection and Dictionary Reconstruction

To handle the domain shift between high-intensity macro-expressions and subtle micro-expressions, the model projects features from both domains into a shared common space. A dictionary of representative patterns is learned for macro-expressions and then used to linearly reconstruct the micro-expression dictionary. This process relies on sparse representation techniques to identify core commonalities.

Sparse patterns used in dictionary learning and reconstruction.

2. Multiview Feature Selection

Rather than relying on a single feature type, the framework extracts four distinct descriptors: LBP-TOP, LOCP-TOP, VLBP, and Optical Flow. To avoid the "curse of dimensionality" and noise associated with simple concatenation, the model employs a multiview representation strategy. Specifically, an $F$-norm block sparse selection process is used to identify the most representative features across these different "views."

Comparison of line sparse and block sparse patterns for feature selection.

Experimental Results

The framework was evaluated using the CK+ database for macro-expressions and the CASME/CASME II databases for micro-expressions, focusing on happiness, sadness, and surprise.

  • Performance: On the CASME II dataset, the framework achieved a recognition rate of 78.2%, outperforming the FDM method (68.5%).
  • Multiview Advantage: The multiview approach (77.0% accuracy) significantly outperformed single-feature baselines like Optical Flow (61.7%), proving that intelligent feature selection is superior to simple stacking.
  • Efficiency: Sparse learning reduced computational overhead while enhancing accuracy by filtering out redundant information.

Conclusion

This multiview transfer learning framework successfully utilizes macro-expression data to overcome the data scarcity in micro-expression research. While the results are state-of-the-art, future work will focus on automating dictionary size selection and improving robustness against real-world environmental factors like lighting and head pose.

Summary updated

Abstract

Multiview representation has become important due to its good performance for machine learning problems. In this paper, a multiview representation framework based on transfer learning is proposed for micro-expression recognition. The framework takes macro-expression as the auxiliary domain and micro-expression as the target domain, and assists the identification of micro-expressions by transferring the rich information extracted from the auxiliary domain, which effectively addresses the small sample problem of micro-expression recognition. The proposed algorithm mainly consists of three parts. Firstly, the features of the two domains are projected into a common space and the dictionaries of each domain are studied respectively. Then the dictionary of micro-expression domain is linearly reconstructed. Finally, in order to improve the comprehensive utilization of feature information, the most representative features from four different micro-expression feature sets are selected by multiview representation. The experiments and evaluation are carried out on three different databases, and the performance comparison of the proposed algorithm with other advanced methods are given. The experimental results show that the proposed algorithm has the better performance than other related methods.
BibTeX
@inproceedings{huang2019multiview,
  author = {Tianhuan Huang and Lei Chen and Yuncong Feng and Xianye Ben and Ruixue Xiao and Tianle Xue},
  title = {{A Multiview Representation Framework for Micro-Expression Recognition}},
  booktitle = {IEEE Access},
  year = {2019},
  doi = {10.1109/access.2019.2932784},
}
Figure