Abstract
Facial Expression Recognition (FER) is a critical component of human-computer interaction, yet it faces challenges due to high-dimensional visual noise and subtle inter-class variations. This paper presents Hy-Facial, a hybrid framework that integrates deep learning features with traditional computer vision descriptors. By fusing global features from VGG19 with local geometric details from SIFT and ORB, and subsequently applying K-means clustering and UMAP dimensionality reduction, the framework achieves a classification accuracy of 83.3% on the FER-Plus dataset. This approach effectively mitigates the "curse of dimensionality" while preserving essential emotional cues.
Introduction
The task of recognizing human emotions from facial images is complicated by the high dimensionality of the data and the subtle differences between expressions, such as the distinction between a neutral face and a slight smirk. Current methodologies typically rely on either deep learning models, which capture high-level global semantics, or handcrafted descriptors like SIFT, which excel at identifying local geometric features. However, simply concatenating these diverse feature sets often leads to redundant, high-dimensional vectors that degrade classifier performance. The Hy-Facial framework addresses this by combining these complementary feature types and employing advanced dimensionality reduction to extract the most salient information.
Methodology
The Hy-Facial pipeline consists of four primary stages designed to optimize feature representation for emotion classification.
- Multi-source Feature Extraction: Global features are extracted using a pre-trained VGG19 network, while local, rotation-invariant details are captured via SIFT (Scale-Invariant Feature Transform) and ORB (Oriented FAST and Rotated BRIEF).
- Feature Selection via Clustering: To manage the resulting data volume, K-means clustering is used to identify "class-wise prototypes." This step filters out noise and reduces redundancy by retaining only the most representative patterns for each emotion.
- Dimensionality Reduction: The authors evaluated six techniques—PCA, t-SNE, UMAP, IsoMap, MDS, and LLE—to compress the hybrid features into a lower-dimensional manifold (optimally 16 dimensions).
- Classification: The refined features are processed by standard classifiers, including Random Forest (RF), K-Nearest Neighbors (KNN), and Multi-Layer Perceptrons (MLP).
The study utilized the FER-Plus dataset, comprising 28,709 training and 3,589 test grayscale images (48x48 pixels) across eight emotion categories.
Results and Discussion
The experimental results demonstrate that the hybridization of global and local features provides a significant performance boost. While VGG19 captures broad emotional categories, SIFT and ORB provide the fine-grained cues necessary to distinguish between similar expressions like "Fear" and "Angry."
As shown in the comparative analysis, UMAP (Uniform Manifold Approximation and Projection) emerged as the superior dimensionality reduction technique. Unlike linear methods like PCA or purely local methods like t-SNE, UMAP preserves both the local and global structures of the facial data. The combination of UMAP and a Random Forest classifier yielded the highest accuracy of 83.3%, a substantial improvement over the 41.85% baseline and standard deep-learning-only approaches.
Conclusion
The Hy-Facial framework proves that fusing deep and traditional features, coupled with intelligent dimensionality reduction, can significantly enhance FER performance. Future research will focus on addressing class imbalances in datasets like FER-Plus and testing the model's robustness in "in-the-wild" scenarios involving occlusions and varying lighting conditions. Additionally, integrating this hybrid approach with Vision Transformers (ViTs) and Explainable AI (XAI) could further improve accuracy and model interpretability.
Summary updated