Optimization in Facial Expression Recognition Based on CNN Combined with SE Modules

Xuanyu ZhangAdvances in Computer Science Research 2024 · DOI · OpenAlex

Abstract

Facial Expression Recognition (FER) in unconstrained environments remains a significant challenge due to variations in lighting, occlusion, and sensor noise. This paper proposes an optimized architecture that integrates Squeeze-and-Excitation (SE) modules into a VGG-like Convolutional Neural Network (CNN). The SE modules enable the network to perform channel-wise feature recalibration, allowing it to focus on the most informative facial features while suppressing noise. Evaluated on the FER-2013 dataset, the proposed model achieves a test accuracy of 61.31%, representing a 2.70% improvement over the baseline CNN.

Introduction

While deep learning has significantly advanced image recognition, FER in "wild" conditions—characterized by inconsistent lighting, overexposure, and low resolution—remains difficult. Standard CNNs often struggle to distinguish subtle emotional cues from background noise because they treat all feature channels with equal importance. This research addresses this limitation by introducing an attention mechanism that prioritizes specific facial features indicative of emotion, such as eyebrow position or mouth shape.

Methodology

The proposed model architecture combines a robust convolutional base with an attention-based recalibration mechanism.

Base Architecture

The network is inspired by the VGG architecture, utilizing multiple convolutional layers with $3 \times 3$ kernels. Each layer is followed by batch normalization, ReLU activation, and max-pooling to extract hierarchical spatial features.

Squeeze-and-Excitation (SE) Modules

To enhance the model's "focus," SE blocks are inserted after every convolutional layer. These blocks operate through three primary stages:

  1. Squeeze: Global Average Pooling compresses spatial information into a channel-wise descriptor.
  2. Excitation: A two-layer fully connected network learns non-linear dependencies between channels.
  3. Scale: The original feature maps are multiplied by the learned weights, effectively amplifying relevant features and attenuating noise.

Data Augmentation

To improve generalization under adverse conditions, the training process included random horizontal flipping, brightness/contrast adjustments, and the addition of Gaussian noise to simulate sensor artifacts.

Datasets

The study utilizes the FER-2013 dataset, a standard benchmark for emotion recognition. The dataset consists of 28,709 training images and 3,589 test images, all in $48 \times 48$ pixel grayscale format. It covers seven emotional categories: angry, disgust, fear, happy, neutral, sad, and surprise.

Example of a 'Surprise' expression from the FER-2013 dataset.

Results and Discussion

The integration of SE modules led to a consistent performance boost across all evaluated metrics. The attention mechanism allowed the model to achieve lower loss and higher accuracy compared to the baseline CNN.

Metric Baseline CNN CNN + SE Attention Improvement
Training Accuracy 61.56% 67.24% +5.68%
Validation Accuracy 55.63% 60.80% +5.17%
Test Accuracy 58.61% 61.31% +2.70%

The results indicate that channel-wise calibration effectively helps the network ignore pixel-level distortions and focus on the structural geometry of the face.

Conclusion

This research demonstrates that adding Squeeze-and-Excitation modules to a standard CNN significantly enhances performance in facial expression recognition tasks. While the addition of SE blocks increases model complexity and computational overhead, the gains in accuracy and noise resilience justify the trade-off. Future work will focus on improving model interpretability through visualization techniques like Grad-CAM and exploring more diverse data augmentation strategies to further bolster robustness.

Summary updated

BibTeX
@inproceedings{zhang2024optimization,
  author = {Xuanyu Zhang},
  title = {{Optimization in Facial Expression Recognition Based on CNN Combined with SE Modules}},
  booktitle = {Advances in Computer Science Research},
  year = {2024},
  doi = {10.2991/978-94-6463-540-9\_99},
}
Figure