Research on Facial Expression Recognition Technology Based on Convolutional-Neural-Network Structure

Junqi Guo, Ke Shan, Hao Wu, Rongfang Bie, Wenwan You, Di LuInt. J. Softw. Innov 2018 · DOI

TL;DR

This paper presents a deep Convolutional Neural Network (CNN) approach for facial expression recognition, aiming to overcome the limitations of traditional methods that rely heavily on manual feature extraction. By leveraging CNNs, the system automatically learns deeper representations of facial expressions, leading to improved accuracy and robustness against noise.

What Problem Does This Paper Address?

Accurately recognizing human facial expressions is a crucial component of Affective Computing, enabling machines to understand and respond to human emotions. Traditional methods for this task often struggle because facial expressions are subtle and can be easily obscured by variations in lighting, pose, and noise. These methods typically require extensive manual feature engineering, which is time-consuming and may not capture the full complexity of expression dynamics. This paper addresses the challenge of developing a more robust and automated facial expression recognition system by exploring the power of deep learning, specifically Convolutional Neural Networks (CNNs).

Methods

The proposed system utilizes a deep CNN architecture for facial expression recognition. The core of the method involves:

  • Input Module: Acquires 2D image data.
  • Pre-processing Module: This module performs two key steps:
  • Face Detection: Identifies and isolates the facial region within an image, likely using methods like Haar-like features.
  • Histogram Equalization: Normalizes image intensity to reduce the impact of varying lighting conditions.
  • Recognition Module: This is the heart of the system, employing a four-layer CNN structure consisting of two convolutional layers and two subsampling layers (C-S-C-S). A Softmax classifier is used for multi-class classification of expressions.
  • Output Module: Monitors the Mean Squared Error (MSE) convergence during training and calculates recognition accuracy. The system iteratively re-adjusts network parameters if the desired accuracy is not met.

The authors also compare their CNN approach with the k-nearest neighbor (KNN) algorithm to validate their findings.

Datasets

The authors use two standard facial expression databases for their simulations and evaluations:

  • Extended Cohn-Kanade Dataset (CK+): Contains expression images from various individuals.
  • Japanese Female Facial Expression Database (JAFFE): Specifically includes images of Japanese women.

Both datasets are categorized into 7 kinds of expressions, including Happy, Sad, Angry, Fear, Surprise, Disgust, and Neutral.

Key Insights and Ideas

  • Automated Feature Learning: The primary insight is that deep CNNs can automatically learn rich and discriminative feature representations of facial expressions, eliminating the need for manual feature engineering. This is a significant advantage over traditional methods.
  • Robustness to Noise: The study investigates the system's anti-noise property by introducing zero-mean Gaussian white noise, suggesting that CNNs can maintain reasonable performance even with degraded input.
  • Expression Pair Analysis: Examining recognition accuracy on different expression pairs helps identify which expressions are more easily confused, providing insights into the challenges of distinguishing similar emotional states.
  • Comparison with KNN: The inclusion of a KNN comparison highlights the superiority of the CNN approach, demonstrating the effectiveness of deep learning for this task.

Key Results

The paper reports on the performance of the proposed CNN system under various factors, including network structure, learning rate, and pre-processing techniques. While specific quantitative accuracy figures are not detailed in the provided text, the authors indicate that they simulate and evaluate the performance, examine anti-noise properties, and discuss confusion issues on similar expression recognition. The comparison with KNN is intended to make the results more convincing, implying that the CNN approach yields superior performance.

Limitations and Future Work

  • Dataset Size and Diversity: While CK+ and JAFFE are standard, the authors might acknowledge that larger and more diverse datasets could further improve generalization.
  • Real-world Complexity: The current system's performance in highly uncontrolled, real-world scenarios with extreme variations in pose, illumination, and occlusions is not explicitly detailed.
  • Computational Cost: Deep CNNs can be computationally intensive, and the paper doesn't detail the training or inference times.

Future work could involve exploring more advanced CNN architectures, incorporating temporal information for dynamic expression analysis, and testing the system on larger and more diverse real-world datasets. The authors also mention examining the influence of different factors on performance, suggesting further fine-tuning and optimization as a potential avenue.

Summary updated

BibTeX
@article{guo2018research,
  author = {Junqi Guo and Ke Shan and Hao Wu and Rongfang Bie and Wenwan You and Di Lu},
  title = {{Research on Facial Expression Recognition Technology Based on Convolutional-Neural-Network Structure}},
  journal = {Int. J. Softw. Innov.},
  year = {2018},
  doi = {10.4018/ijsi.2018100108},
}
Figure