Abstract
Facial Expression Recognition (FER) often prioritizes visual textures over underlying spatial structures, leading to performance degradation in "in-the-wild" scenarios. We present CS-GBSBF (Component Separation and Granular-ball Space Bootstrap Fusion), a novel framework that converts images into granular-ball spatial graphs. By fusing traditional visual features with these structural representations and employing a component-based separation strategy, the model achieves state-of-the-art accuracy across eight major datasets, including a 97.34% accuracy on Oulu-CASIA.
Introduction
While deep learning has advanced computer vision, FER remains challenging due to variability in lighting, pose, and identity. Most current methods focus on pixel-level textures, often discarding the morphological changes—muscle movements and feature shifts—that define expressions. While facial landmarks can capture geometry, they are labor-intensive and often too sparse to represent the full complexity of facial dynamics.
The proposed method addresses this by capturing "silent signals" of facial geometry without losing rich visual detail. This is achieved through a "granular-ball" representation that clusters similar pixel regions into graph nodes, preserving topology and local densities while reducing redundant data.
The CS-GBSBF Framework
The Component Separation and Granular-ball Space Bootstrap Fusion (CS-GBSBF) architecture consists of three primary pillars:
- Granular-ball Representation: Images are transformed into spatial graphs where nodes represent clusters of similar pixels. This preserves relative positions and spatial topology while reducing data redundancy compared to standard pixel grids.
- Represent Extraction & Separation (REN & RSN): A two-stream approach processes raw visual images and spatial granular-ball graphs separately. The RSN decomposes these global features into specific facial components (e.g., eyes, nose, mouth), allowing for targeted analysis of expressive regions.
- Bootstrap Fusion (RFN): To align the disparate visual and spatial streams, an Attention Guidance Module (AGM) and a Bootstrap Alignment Loss ($L_{BA}$) are used. These ensure spatial consistency, forcing the model to correlate visual features of a component (like the mouth) with its corresponding spatial graph representation.
Key Insights
- Structural Superiority: Converting images to graphs via granular balls captures angular direction and spatial topology that standard Convolutional Neural Networks (CNNs) often miss.
- Adaptive Component Weighting: By separating facial components, the model learns to weight regions like the eyes or mouth more heavily for specific expressions, ignoring static or irrelevant areas.
- Alignment via Bootstrap Loss: The alignment gap between pixel and graph streams is bridged by the Bootstrap Alignment Loss, which synchronizes the two representations to ensure they are analyzing the same facial features.
Experimental Results
The CS-GBSBF method was evaluated on eight datasets, including "in-the-wild" (AffectNet-8, RAF-DB, FER-2013) and lab-controlled (Oulu-CASIA, CK+) environments.
- State-of-the-Art Performance: The model achieved 97.34% accuracy on Oulu-CASIA and 96.74% on the SAMM micro-expression dataset, proving its sensitivity to fleeting movements.
- Generalization: Across all tested databases, the method provided an average accuracy improvement of 2.48% over existing SOTA models.
- Ablation Studies: Removing the spatial graph stream or the alignment loss resulted in an average performance drop of 1.93%, confirming the necessity of the dual-stream granular-ball approach.
Conclusion and Future Work
CS-GBSBF demonstrates that integrating spatial topology with visual features significantly enhances FER robustness. While the current model focuses on static images, future work will explore extending the granular-ball graph approach to video sequences to capture the temporal velocity of muscle movements, potentially further improving micro-expression detection.
Summary updated