A False Trail to Follow: Differential Effects of the Facial Feedback Signals From the Upper and Lower Face on the Recognition of Micro-Expressions

Xuemei Zeng, Qi Wu, Siwei Zhang, Zheying Liu, Qing Zhou, Meishan ZhangFrontiers in Psychology 2018 · DOI

TL;DR

This study reveals that facial feedback, the subtle muscle movements we make when perceiving an expression, plays a differential role in recognizing micro-expressions. Specifically, enhancing feedback from the upper face can improve recognition of intense micro-expressions, while enhancing feedback from the lower face impairs recognition of subtle micro-expressions. Blocking lower face feedback consistently boosts micro-expression recognition, suggesting the lower face is a "false trail" for these rapid emotional signals.

What Problem Does This Paper Address?

Micro-expressions are incredibly brief (under 0.5 seconds) facial movements that can betray true emotions, making them crucial for lie detection and understanding subtle social cues. However, their speed and subtlety make them notoriously difficult to recognize. While training can improve recognition, its real-world effectiveness is limited. To develop better training and understanding, researchers need to delve into the cognitive mechanisms underlying micro-expression perception. This paper addresses the question of whether facial feedback – the proprioceptive signals generated by our own facial muscles mimicking a perceived expression – influences micro-expression recognition, and if so, whether different parts of the face play distinct roles. Existing research on macro-expressions (longer, more obvious expressions) suggests facial feedback is important, but its role in the rapid, fleeting world of micro-expressions remained unclear, particularly regarding the specific contributions of the upper versus lower face.

Methods

The researchers employed a series of behavioral experiments to investigate the impact of manipulating facial feedback on micro-expression recognition.

  • Facial Feedback Manipulation: A key technique involved using a restricting gel (polyvinyl alcohol and polyvinylpyrrolidone) that, when applied to specific facial regions (upper face, lower face, or full face), amplified proprioceptive feedback by creating resistance during muscle movements. Conversely, a control condition used a moisturizer. In Study 3, a mouthguard was used to actively block lower face feedback.
  • Micro-Expression Recognition Task: Participants viewed short video clips of micro-expressions from the BU-3DFE database. These expressions were presented using the JACBART paradigm, which involves sandwiching the target expression between neutral faces of the same individual. Participants then identified the emotion displayed from a list of options.
  • Stimuli: The BU-3DFE database provided facial expressions varying in intensity (subtle vs. intense) and duration (50 ms to 450 ms).
  • Control Task: To rule out confounding effects of cognitive load, a working memory task (modular arithmetic questions) was included.

The studies used a mixed-model experimental design, varying the facial feedback manipulation (between-subjects) and expression duration/intensity (within-subjects).

Datasets

The study utilized the BU-3DFE database. This database contains 100 models with diverse ethnicities and ages, posing six basic facial expressions (sadness, surprise, anger, disgust, fear, happiness) at four intensity levels. The researchers selected specific expressions based on their intensity (subtle or intense) and duration for their experiments. No new benchmarks were introduced; the focus was on evaluating existing micro-expression recognition paradigms.

Key Insights and Ideas

  • The Face is Not Monolithic: The most striking insight is that different facial regions contribute differently to micro-expression recognition. This challenges a uniform view of facial feedback and highlights the importance of spatial specificity.
  • Lower Face as a "False Trail": The lower face appears to be a detrimental source of feedback for micro-expressions. Amplifying its feedback impairs recognition, while blocking it improves it. This suggests that the signals from the lower face, perhaps due to its more voluntary control and perceptual weight, can actively mislead the observer when processing rapid emotional cues.
  • Upper Face as a Beneficial Cue (Under Conditions): Enhancing upper face feedback showed a positive effect, but only for intense micro-expressions and when they were presented for a longer duration (450 ms). This suggests the upper face's feedback is more subtle and requires specific conditions to be beneficial.
  • Intensity Matters: The impact of facial feedback is modulated by the intensity of the micro-expression. Subtle expressions are more susceptible to the detrimental effects of lower face feedback, while intense expressions can benefit from upper face feedback under certain conditions.
  • Embodied Cognition Extended: These findings extend the embodied theory of cognition to the realm of micro-expressions, demonstrating that our own bodily feedback plays a role even in perceiving these extremely rapid and subtle emotional signals.

Key Results

  • Study 1 (Intense Micro-Expressions): Amplifying upper face feedback significantly improved recognition accuracy for intense micro-expressions at a duration of 450 ms. Amplifying lower face feedback did not yield significant effects for intense micro-expressions.
  • Study 2 (Subtle Micro-Expressions): Amplifying lower face feedback significantly impaired recognition accuracy for subtle micro-expressions across all tested durations (50, 150, 333, and 450 ms). Amplifying upper face feedback did not significantly affect subtle micro-expression recognition.
  • Study 3 (Blocking Lower Face Feedback): Blocking lower face feedback (using a mouthguard) significantly boosted recognition accuracy for both subtle and intense micro-expressions at durations of 150 ms and 333 ms. This provides strong support for the detrimental role of lower face feedback.
  • Working Memory: In all studies, facial feedback manipulation did not significantly affect performance on a working memory task, indicating that the observed effects were specific to micro-expression recognition and not due to general cognitive load.

Limitations and Future Work

  • Stimulus Ecological Validity: The study used the JACBART paradigm, which employs static images. Future research should explore these effects with more dynamic and naturalistic micro-expression stimuli.
  • Emotion Category Specificity: While the study focused on overall recognition, the authors acknowledge that specific emotion categories might be differentially affected by facial feedback. Future work could investigate this with more trials per emotion.
  • Mechanisms of Differential Effects: The exact neural and motor mechanisms underlying the differential effects of upper and lower face feedback remain unclear. The authors suggest exploring voluntary vs. spontaneous muscle control and perceptual weighting as potential explanations.
  • Higher-Level Processing: The study used a verbal judgment task. Future research could investigate at which stage of processing (visual vs. conceptual) facial feedback exerts its influence on micro-expression perception.
  • Action Unit Specificity: The role of specific facial Action Units (AUs) in mediating these effects warrants further investigation.

Summary updated

Abstract

Micro-expressions, as fleeting facial expressions, are very important for judging people's true emotions, thus can provide an essential behavioral clue for lie and dangerous demeanor detection. From embodied accounts of cognition, we derived a novel hypothesis that facial feedback from upper and lower facial regions has differential effects on micro-expression recognition. This hypothesis was tested and supported across three studies. Specifically, the results of Study 1 showed that people became better judges of intense micro-expressions with a duration of 450 ms when the facial feedback from upper face was enhanced via a restricting gel. Additional results of Study 2 showed that the recognition accuracy of subtle micro-expressions was significantly impaired under all duration conditions (50, 150, 333, and 450 ms) when facial feedback from lower face was enhanced. In addition, the results of Study 3 also revealed that blocking the facial feedback of lower face, significantly boosted the recognition accuracy of subtle and intense micro-expressions under all duration conditions (150 and 450 ms). Together, these results highlight the role of facial feedback in judging the subtle movements of micro-expressions.
BibTeX
@inproceedings{zeng2018false,
  author = {Xuemei Zeng and Qi Wu and Siwei Zhang and Zheying Liu and Qing Zhou and Meishan Zhang},
  title = {{A False Trail to Follow: Differential Effects of the Facial Feedback Signals From the Upper and Lower Face on the Recognition of Micro-Expressions}},
  booktitle = {Frontiers in Psychology},
  year = {2018},
  doi = {10.3389/fpsyg.2018.02015},
}
Figure