Shape-invariant encoding of dynamic primate facial expressions in human perception

Nick Taubert, Michael Stettler, Ramona Siebert, Silvia Spadacenta, Louisa Sting, Peter W. Dicke, Peter Thier, Martin A. GieseeLife 2021 · DOI

TL;DR

Humans can quickly learn and recognize dynamic facial expressions across primate species, largely independent of the specific face shape. This suggests that our brains process the dynamics of expressions separately from the static features of the face, challenging models that rely solely on appearance.

What Problem Does This Paper Address?

Understanding how we perceive dynamic facial expressions is crucial for social communication, but it's tricky to study. Previous research often used static images, missing the crucial element of movement. When dynamic expressions were studied, it was hard to precisely control the shape and timing of the expressions, especially across different species like humans and monkeys. This made it unclear whether our brains encode expressions based on their appearance (shape) or their movement dynamics, and how this might differ between our own species and others. The authors hypothesize that due to the slower evolution of facial musculature compared to facial shape, expression dynamics might be processed independently of facial shape.

Methods

The researchers developed a sophisticated approach combining computer animation and machine learning. They created highly realistic 3D avatars of human and monkey faces. The movement data for these avatars came from motion capture of real human and monkey facial expressions. A key innovation was using a Bayesian machine-learning technique to create a "morphing space" that allowed for precise, continuous control over expression dynamics. This enabled them to generate a set of stimuli that smoothly interpolated between different expression types (anger/threat vs. fear) and species (human vs. monkey). Participants then viewed these animated faces and were asked to classify the expression type and species. They also conducted control experiments to test the robustness of their findings by occluding ear regions and equilibrating the low-level motion features of the stimuli.

Datasets

The study utilized motion capture data from real human and monkey facial expressions as the basis for their animated stimuli. No new benchmark datasets were explicitly introduced, but the generated stimuli formed a novel, controlled dataset for studying cross-species facial expression perception.

Key Insights and Ideas

  • Fast Cross-Species Learning: Humans can learn to recognize monkey facial expressions surprisingly quickly, even without prior exposure. This suggests a shared underlying mechanism for processing dynamic expressions across primates.
  • Shape-Invariant Perception: The most striking finding is that human observers largely perceived the dynamics of facial expressions independently of the shape of the face (human vs. monkey avatar). This means the movement itself, not the specific face it's on, is the primary cue.
  • Dynamics Over Appearance: This independence challenges current neural network models that might rely heavily on static appearance features for expression recognition. It points towards a processing stream that prioritizes dynamic cues.
  • Own-Species Advantage: While cross-species recognition is good, there's still a subtle advantage for recognizing expressions from one's own species, indicated by narrower "tuning" for human expressions.

Key Results

  • Participants learned to categorize cross-species expressions very quickly, requiring only a few repetitions.
  • The accuracy of classifying expression dynamics was largely unaffected by whether the avatar's species matched the expression's origin (e.g., recognizing a monkey expression on a monkey avatar vs. a human avatar).
  • Species-specific expression tuning was narrower for human expressions compared to monkey expressions, suggesting a slight own-species advantage.
  • Control experiments (occluding ears, equilibrating motion features) confirmed the robustness of these findings, indicating that specific species-specific features or low-level motion cues weren't solely responsible for the results.

Limitations and Future Work

The authors acknowledge that their study focused on two specific expression types (anger/threat and fear) and two primate species (human and macaque monkey). Future work could explore a wider range of expressions and species, including non-primate faces, to test the generality of the shape-invariant hypothesis. They also suggest that their technology for generating realistic, controlled dynamic facial stimuli opens doors for electrophysiological studies to uncover the underlying neural mechanisms of this shape-independent expression encoding.

Summary updated

Abstract

Dynamic facial expressions are crucial for communication in primates. Due to the difficulty to control shape and dynamics of facial expressions across species, it is unknown how species-specific facial expressions are perceptually encoded and interact with the representation of facial shape. While popular neural network models predict a joint encoding of facial shape and dynamics, the neuromuscular control of faces evolved more slowly than facial shape, suggesting a separate encoding. To investigate these alternative hypotheses, we developed photo-realistic human and monkey heads that were animated with motion capture data from monkeys and humans. Exact control of expression dynamics was accomplished by a Bayesian machine-learning technique. Consistent with our hypothesis, we found that human observers learned cross-species expressions very quickly, where face dynamics was represented largely independently of facial shape. This result supports the co-evolution of the visual processing and motor control of facial expressions, while it challenges appearance-based neural network theories of dynamic expression recognition.
BibTeX
@inproceedings{taubert2021shape,
  author = {Nick Taubert and Michael Stettler and Ramona Siebert and Silvia Spadacenta and Louisa Sting and Peter W. Dicke and Peter Thier and Martin A. Giese},
  title = {{Shape-invariant encoding of dynamic primate facial expressions in human perception}},
  booktitle = {eLife},
  year = {2021},
  doi = {10.7554/elife.61197},
}
Figure