The Influence of Key Facial Features on Recognition of Emotion in Cartoon Faces

Shu Zhang, Xinge Liu, Xuan Yang, Yezhi Shu, Niqi Liu, Dan Zhang, Yong‐Jin LiuFrontiers in Psychology 2021 · DOI

TL;DR

This study investigates how humans recognize emotions in cartoon faces, finding that a "happiness advantage" exists, similar to real faces. Crucially, it reveals that the mouth is key for recognizing happiness, while eyebrows are essential for sadness, offering insights for creating more emotionally expressive AI characters.

What Problem Does This Paper Address?

Cartoon characters are ubiquitous in digital media and increasingly integrated into social robots and virtual agents. While they effectively convey emotions, the underlying mechanisms of how humans recognize these emotions in stylized cartoon faces remain unclear. This lack of understanding hinders the development of more sophisticated and emotionally intelligent human-computer interactions. Existing research on real faces suggests certain features are more critical for specific emotions, but it's unknown if these principles translate to the exaggerated and simplified nature of cartoon faces. This paper aims to bridge this gap by systematically exploring emotion recognition in cartoon faces and identifying the specific facial features that drive this perception.

Methods

The researchers conducted three experiments using a combination of real and cartoon facial stimuli.

  • Experiment 1: Compared the recognition accuracy and perceived intensity of emotions (happy, sad, neutral) in full cartoon faces versus full real faces.
  • Experiment 2: Investigated the "sufficiency" of individual facial features (mouth, eyes, eyebrows) for emotion recognition. This involved presenting only one feature at a time (with others blurred) and comparing recognition accuracy to the full-face condition.
  • Experiment 3: Examined the "necessity" of individual facial features by hiding one feature at a time and assessing the impact on recognition accuracy and perceived intensity compared to the full-face condition.

All experiments used a within-participants design, and participants were asked to categorize the emotion and rate its intensity. Statistical analyses, including repeated-measures ANOVAs, were used to determine the significance of various effects.

Datasets

The study utilized facial expression images converted from a database of real faces posed by young Chinese women. These real faces were then transformed into Japanese-style cartoon faces using a U-GAT-IT computational model. The quality of the cartoon expressions was further refined by an artist. The participants were Chinese students from Tsinghua University.

Key Insights and Ideas

  • Cartoon Faces Aren't Just Simplified Real Faces: While cartoon faces share some recognition patterns with real faces (like the happiness advantage), they also exhibit unique characteristics. For instance, sad cartoon expressions are perceived as sadder than their real-life counterparts.
  • Feature Focus for Specific Emotions: The study strongly suggests a feature-based model for cartoon emotion recognition. The mouth is identified as both sufficient and necessary for recognizing happiness. Conversely, eyebrows are found to be sufficient and necessary for the perceived intensity of sadness.
  • The "Happiness Advantage" Holds: Similar to real faces, happy cartoon expressions were recognized more accurately than neutral and sad ones. This suggests that even with stylization, the fundamental advantage of recognizing happiness persists.
  • Sadness Intensity is Amplified in Cartoons: Cartoon faces, particularly when depicting sadness, seem to convey a stronger emotional intensity compared to real faces. This could be due to the exaggerated nature of cartoon features.

Key Results

  • Accuracy: Happy cartoon expressions were recognized more accurately than neutral and sad ones. Sad cartoon expressions were recognized more accurately than sad real faces.
  • Intensity: Sad cartoon expressions were perceived as sadder than sad real expressions. Happy cartoon expressions were perceived as less happy than happy real expressions.
  • Mouth for Happiness: Presenting only the mouth was sufficient for recognizing happiness in cartoon faces, and removing the mouth significantly impaired recognition accuracy.
  • Eyebrows for Sadness: Eyebrows were found to be necessary for the perceived intensity of sad expressions in cartoon faces.

Limitations and Future Work

The authors acknowledge several limitations:

  • Limited Emotion Set: The study only included happy, sad, and neutral emotions. Future work should explore other basic emotions (anger, fear, surprise, disgust) to see if the feature contributions generalize.
  • Cultural Specificity: The study used Chinese participants and Japanese-style cartoon faces. Generalizability to other cultures, especially those with different facial expression interpretation norms (e.g., East Asian cultures focusing more on the eye region), needs further investigation.
  • Behavioral Data Only: The study relied solely on behavioral responses. Incorporating psychophysiological data (like eye-tracking or EEG) would provide deeper insights into the processing mechanisms.
  • Computational Model and Artist Refinement: While efforts were made to ensure cartoon expressions matched real ones, the exact equivalence of emotional information conveyed by stylized vs. realistic features remains a question.
  • Generalizability of Feature Masking: The use of Gaussian blur for masking features is standard, but its real-world applicability to situations with occlusions (like masks) warrants further study.

Future directions include exploring the interplay of multiple facial features, investigating different levels of cartoon stylization, and examining the role of genuineness in emotion perception.

Summary updated

Abstract

Cartoon faces are widely used in social media, animation production, and social robots because of their attractive ability to convey different emotional information. Despite their popular applications, the mechanisms of recognizing emotional expressions in cartoon faces are still unclear. Therefore, three experiments were conducted in this study to systematically explore a recognition process for emotional cartoon expressions (happy, sad, and neutral) and to examine the influence of key facial features (mouth, eyes, and eyebrows) on emotion recognition. Across the experiments, three presentation conditions were employed: (1) a full face; (2) individual feature only (with two other features concealed); and (3) one feature concealed with two other features presented. The cartoon face images used in this study were converted from a set of real faces acted by Chinese posers, and the observers were Chinese. The results show that happy cartoon expressions were recognized more accurately than neutral and sad expressions, which was consistent with the happiness recognition advantage revealed in real face studies. Compared with real facial expressions, sad cartoon expressions were perceived as sadder, and happy cartoon expressions were perceived as less happy, regardless of whether full-face or single facial features were viewed. For cartoon faces, the mouth was demonstrated to be a feature that is sufficient and necessary for the recognition of happiness, and the eyebrows were sufficient and necessary for the recognition of sadness. This study helps to clarify the perception mechanism underlying emotion recognition in cartoon faces and sheds some light on directions for future research on intelligent human-computer interactions.
BibTeX
@inproceedings{zhang2021influence,
  author = {Shu Zhang and Xinge Liu and Xuan Yang and Yezhi Shu and Niqi Liu and Dan Zhang and Yong‐Jin Liu},
  title = {{The Influence of Key Facial Features on Recognition of Emotion in Cartoon Faces}},
  booktitle = {Frontiers in Psychology},
  year = {2021},
  doi = {10.3389/fpsyg.2021.687974},
}
Figure