Dynamic Facial Expression of Fuzzy Modeling Using Probability of Emotion

Hyo-Seok Kang, Jaeho Baek, Euntai Kim, Mignon ParkJournal of Korean Institute of Intelligent Systems 2009 · DOI

Abstract

This paper suggests applying a mirror-reflected method based on a 2D emotion recognition database to 3D applications. It generates facial expressions using fuzzy modeling based on the probability of emotion. The suggested facial expression function applies fuzzy theory to three basic movements for facial expressions. This method utilizes mirror-reflected multi-images to apply feature vectors for emotion recognition from 2D applications to 3D models. Consequently, a model based on fuzzy non-linear facial expressions of a real model can be achieved. The system uses average values for the probabilities of six basic expressions: happy, sad, disgust, angry, surprise, and fear. Dynamic facial expressions are generated via fuzzy modeling, and the paper compares and analyzes the feature vectors of a real model with a 3D human-like avatar.

Introduction

While 2D facial expression recognition is a mature field, translating these expressions into realistic 3D avatars remains a significant challenge. Human facial expressions are inherently non-linear; transitions between emotions do not follow simple linear paths. Traditional systems often rely on discrete states or linear interpolation, which can result in "robotic" or unnatural movements. This research addresses the gap in accurately mapping the subtle, non-linear dynamics of human facial movement from 2D data into 3D space using fuzzy logic.

Methodology

3D Feature Extraction via Mirror Reflection

To capture 3D coordinates without the complexity of multi-camera synchronization, the researchers employed a mirror-reflection setup. By using a single real camera and two mirrors, they created "virtual cameras" that allowed for the capture of multi-view images in a single 2D frame. This setup enables the calculation of 3D feature coordinates from a single synchronized source, bridging the gap between 2D image data and 3D modeling.

Mirror reflection setup for 3D feature extraction from a single camera.

Feature Point Selection

The model utilizes 19 specific Facial Animation Parameters (FAPs) based on the MPEG-4 standard. These points are concentrated on the most expressive regions of the face, including the eyes, eyebrows, and mouth, which are critical for conveying emotional nuance.

The 19 selected Facial Animation Parameters (FAPs) used for modeling.

Fuzzy Inference System

The core of the proposed method is a Fuzzy Inference System (FIS). Unlike hard-coded binary rules, fuzzy logic handles the "gray areas" of human emotion:

  • Inputs: Probabilities of six basic emotions (Happy, Sad, Disgust, Angry, Surprise, Fear).
  • Processing: A "Table Look-up" method defines fuzzy rules based on experimental data, mapping emotion intensities to physical facial changes.
  • Output: The system calculates the physical ratios and sizes of facial features.
  • Defuzzification: The Center of Gravity method converts fuzzy results into crisp numerical values to drive the 3D avatar's mesh.

Results and Discussion

The study utilized standard 2D emotion databases (such as Kanade-Cohn) alongside a custom 3D dataset generated via the mirror-reflection setup. Key findings include:

  • Non-linear Dynamics: The use of fuzzy logic successfully modeled the non-linear nature of skin and muscle movement, providing a more realistic representation than linear models.
  • Probability-Driven Transitions: By using emotion probability as an input, the system generates smooth transitions between states. This avoids the jerky movements seen in systems that switch between discrete emotional labels.
  • Accuracy and Realism: Comparisons between the generated avatar expressions and real human feature vectors demonstrated a significantly lower error rate for the fuzzy model. Qualitative analysis showed that the 3D avatar successfully demonstrated complex, dynamic expressions like "Fear" and "Joy" that looked natural during the movement phase.

Conclusion

The proposed method demonstrates that 2D emotion data can be effectively leveraged for 3D modeling through mirror reflection and fuzzy logic. While the current model is limited to 19 feature points and six basic emotions, it provides a foundation for more lifelike digital humans. Future work will focus on increasing mesh density and incorporating more complex, blended emotional states.

This paper was originally published in the Journal of Korea Institute of Intelligent Systems (2009).

Summary updated

Abstract

본 논문은 거울 투영을 이용하여 2D의 감정인식 데이터베이스를 3D에 적용 가능하다는 것을 증명한다. 또한, 감정 확률을 이용하여 퍼지 모델링 기반의 얼굴표정을 생성하고, 표정을 움직이는 3가지 기본 움직임에 대한 퍼지이론을 적용하여 얼굴표현함수를 제안한다. 제안된 방법은 거울 투영을 통한 다중 이미지를 이용하여 2D에서 사용되는 감정인식에 대한 특징벡터를 3D에 적용한다. 이로 인해, 2D의 모델링 대상이 되는 실제 모델의 기본감정에 대한 비선형적인 얼굴표정을 퍼지를 기반으로 모델링한다. 그리고 얼굴표정을 표현하는데 기본 감정 5가지인 행복, 슬픔, 혐오, 화남, 놀람, 무서움으로 표현되며 기본 감정의 확률에 대해서 각 감정의 평균값을 사용하고 6가지 감정 확률을 이용하여 동적 얼굴표정을 생성한다. 제안된 방법을 3D 인간형 아바타에 적용하여 실제 모델의 표정 벡터와 비교 분석한다. This paper suggests to apply mirror-reflected method based 2D emotion recognition database to 3D application. Also, it makes facial expression of fuzzy modeling using probability of emotion. Suggested facial expression function applies fuzzy theory to 3 basic movement for facial expressions. This method applies 3D application to feature vector for emotion recognition from 2D application using mirror-reflected multi-image. Thus, we can have model based on fuzzy nonlinear facial expression of a 2D model for a real model. We use average values about probability of 6 basic expressions such as happy, sad, disgust, angry, surprise and fear. Furthermore, dynimic facial expressions are made via fuzzy modelling. This paper compares and analyzes feature vectors of real model with 3D human-like avatar.
BibTeX
@article{kang2009dynamic,
  author = {Hyo-Seok Kang and Jaeho Baek and Euntai Kim and Mignon Park},
  title = {{Dynamic Facial Expression of Fuzzy Modeling Using Probability of Emotion}},
  journal = {Journal of Korean Institute of Intelligent Systems},
  year = {2009},
  doi = {10.5391/jkiis.2009.19.1.001},
}
Figure