Abstract
This paper suggests applying a mirror-reflected method based on a 2D emotion recognition database to 3D applications. It generates facial expressions using fuzzy modeling based on the probability of emotion. The suggested facial expression function applies fuzzy theory to three basic movements for facial expressions. This method utilizes mirror-reflected multi-images to apply feature vectors for emotion recognition from 2D applications to 3D models. Consequently, a model based on fuzzy non-linear facial expressions of a real model can be achieved. The system uses average values for the probabilities of six basic expressions: happy, sad, disgust, angry, surprise, and fear. Dynamic facial expressions are generated via fuzzy modeling, and the paper compares and analyzes the feature vectors of a real model with a 3D human-like avatar.
Introduction
While 2D facial expression recognition is a mature field, translating these expressions into realistic 3D avatars remains a significant challenge. Human facial expressions are inherently non-linear; transitions between emotions do not follow simple linear paths. Traditional systems often rely on discrete states or linear interpolation, which can result in "robotic" or unnatural movements. This research addresses the gap in accurately mapping the subtle, non-linear dynamics of human facial movement from 2D data into 3D space using fuzzy logic.
Methodology
3D Feature Extraction via Mirror Reflection
To capture 3D coordinates without the complexity of multi-camera synchronization, the researchers employed a mirror-reflection setup. By using a single real camera and two mirrors, they created "virtual cameras" that allowed for the capture of multi-view images in a single 2D frame. This setup enables the calculation of 3D feature coordinates from a single synchronized source, bridging the gap between 2D image data and 3D modeling.
Feature Point Selection
The model utilizes 19 specific Facial Animation Parameters (FAPs) based on the MPEG-4 standard. These points are concentrated on the most expressive regions of the face, including the eyes, eyebrows, and mouth, which are critical for conveying emotional nuance.
Fuzzy Inference System
The core of the proposed method is a Fuzzy Inference System (FIS). Unlike hard-coded binary rules, fuzzy logic handles the "gray areas" of human emotion:
- Inputs: Probabilities of six basic emotions (Happy, Sad, Disgust, Angry, Surprise, Fear).
- Processing: A "Table Look-up" method defines fuzzy rules based on experimental data, mapping emotion intensities to physical facial changes.
- Output: The system calculates the physical ratios and sizes of facial features.
- Defuzzification: The Center of Gravity method converts fuzzy results into crisp numerical values to drive the 3D avatar's mesh.
Results and Discussion
The study utilized standard 2D emotion databases (such as Kanade-Cohn) alongside a custom 3D dataset generated via the mirror-reflection setup. Key findings include:
- Non-linear Dynamics: The use of fuzzy logic successfully modeled the non-linear nature of skin and muscle movement, providing a more realistic representation than linear models.
- Probability-Driven Transitions: By using emotion probability as an input, the system generates smooth transitions between states. This avoids the jerky movements seen in systems that switch between discrete emotional labels.
- Accuracy and Realism: Comparisons between the generated avatar expressions and real human feature vectors demonstrated a significantly lower error rate for the fuzzy model. Qualitative analysis showed that the 3D avatar successfully demonstrated complex, dynamic expressions like "Fear" and "Joy" that looked natural during the movement phase.
Conclusion
The proposed method demonstrates that 2D emotion data can be effectively leveraged for 3D modeling through mirror reflection and fuzzy logic. While the current model is limited to 19 feature points and six basic emotions, it provides a foundation for more lifelike digital humans. Future work will focus on increasing mesh density and incorporating more complex, blended emotional states.
This paper was originally published in the Journal of Korea Institute of Intelligent Systems (2009).
Summary updated