Abstract
본 논문에서는 6개의 특징점을 이용하는 가버 웨이블릿 신경망 기반 적응 표정인식 시스템을 제안한다. 특징 추출부를 포함하는 초기 네트워크의 구성은 Levenberg-Marquardt 기반의 학습방법이 사용되며, 따라서 특징 추출부 결정에 있어서 경험적 요소를 배재시킬 수 있다. 또한 새로운 사용자에 대한 적응 네트워크를 구성하기 위해서 개선된 보상함수를 가지는 Q-학습과, 비지도 퍼지 신경망 모델을 사용하였다. Q-학습을 통해서는 개인 사용자에 대해 분리도가 좋은 특징벡터를 얻을 수 있는 가버필터 세트를 얻을 수 있으며, 퍼지 신경망을 통해서는 사용자의 얼굴변화에 맞게 인식기를 변화시킬 수 있다. 따라서 제안된 시스템은 사용자의 얼굴변화를 따라갈 수 있는 좋은 적응 성능을 보이고 있다.
This paper proposes an adaptive Facial Emotional Recognition system based on a Gabor Wavelet Neural Network (GWNN), utilizing six feature points to extract specific facial expression features. A Levenberg-Marquardt-based training methodology is employed to formulate the initial network, excluding heuristics in the feature extraction process. To create an adaptive network for new users, Q-learning with an enhanced reward function and an unsupervised fuzzy neural network model are used. Q-learning enables the system to obtain optimal Gabor filter sets for separable features, while the Fuzzy Neural Network allows adaptation to user changes. The proposed system demonstrates robust online adaptation capabilities, continuously tracing changes in the user's face.
Introduction
Facial expression recognition is a complex task due to the high variability between individuals and the "curse of dimensionality." General models trained on broad databases often struggle when applied to specific users whose facial structures or emotional expressions deviate from the training set. Furthermore, traditional systems often rely on hand-tuned heuristics for feature extraction, limiting their flexibility and ability to adapt once deployed.
This paper addresses these challenges by introducing a system that initializes without manual tuning and personalizes its recognition capabilities through online learning. By combining Gabor Wavelet Neural Networks with reinforcement learning and fuzzy logic, the system can fine-tune its feature extractors in real-time, significantly boosting accuracy for new users compared to static, pre-trained models.
System Architecture
The core of the proposed system is the Gabor Wavelet Neural Network (GWNN). Gabor filters are mathematical functions that mimic human visual processing, making them ideal for extracting features from facial images. The system focuses on six key feature points on the face to capture the nuances of expression.
To avoid the limitations of manual parameter setting, the authors utilize the Levenberg-Marquardt (LM) algorithm. This optimization technique trains the entire network—including the feature extraction layer—simultaneously. This data-driven approach ensures that the initial Gabor filters are optimized for general expression recognition without requiring expert intuition.
Adaptive Personalization Mechanism
The true strength of the system lies in its ability to adapt to a specific user. This is achieved through a two-pronged approach:
- Q-Learning for Filter Optimization: The system treats the adjustment of Gabor filter parameters as "actions" within a reinforcement learning framework. Using a custom reward function, Q-learning identifies the specific filter set that maximizes the separability of different expressions for a particular individual.
- Unsupervised Fuzzy Neural Network: For final classification, an unsupervised fuzzy network (similar to a Self-Organizing Map) clusters the extracted features. This allows the system to continuously track and adapt to subtle changes in a user’s facial expressions over time, such as those caused by lighting or aging, without requiring constant labeled feedback.
Experimental Results
The system was evaluated using the EKMAN and JAFFE databases for initial training, achieving accuracies of 74% and 89%, respectively. However, the most significant results were observed in personalization tests using a custom local database (BSCL) of 600 images from five specific users.
In one instance, a general model that initially achieved only 33.3% accuracy for a specific user improved to 100% accuracy after the Q-learning and fuzzy adaptation process. Across all test users, the final adapted accuracy ranged from 78.3% to 100%. Additionally, the system demonstrated real-time efficiency, processing 10 frames per second on standard hardware and successfully running on a PDA.
Conclusion
The proposed adaptive facial expression recognition system effectively overcomes the limitations of static, heuristic-based models. By integrating LM training, Q-learning, and unsupervised fuzzy logic, the system provides a robust framework for personalized, real-time emotion recognition. Future work will focus on expanding the recognizable emotion set beyond happiness, sadness, and anger, and further automating the initial feature point selection process.
Summary updated