Abstract
Facial Expression Recognition (FER) remains a challenging task in human-computer interaction due to the subtle and holistic nature of emotional manifestations. We present Inversion FER (IFER), a novel framework that enhances expression recognition by integrating facial image inversion. At its core is the Adversarial Style Inversion Transformer (ASIT), which utilizes long-range dependencies to capture both global facial structures and local details. Our approach demonstrates that the process of reconstructing a face provides a rich feature set for classification, leading to state-of-the-art performance across multiple benchmarks.
Introduction
Traditional FER models primarily function as discriminative classifiers, often focusing on isolated facial patches like the eyes or mouth. However, emotions are expressed through complex, global changes in facial geometry. Current methods struggle to bridge the gap between local feature extraction and global context. We propose that the underlying structural information used in image generation can significantly bolster recognition accuracy. By learning to "invert" an image—mapping it back to a generative latent space—a model gains a deeper understanding of facial semantics.
Methodology
The IFER framework consists of two main pillars: the ASIT architecture and the inversion-based recognition task.
Adversarial Style Inversion Transformer (ASIT)
Unlike standard CNN-based inversion models, ASIT employs a pure Transformer architecture. This design allows for the modeling of long-range dependencies, ensuring the system considers the entire face simultaneously rather than just local neighborhoods. The model uses a window-based attention mechanism that transitions from a broad global view to fine-grained local details.
Learning through Inversion
To ensure semantic fidelity, we utilize an Image Inversion Discriminator that measures cosine similarity between source and generated features. Furthermore, a Distribution Alignment Loss is applied, treating a pre-trained StyleGAN2 generator as a teacher to guide the ASIT encoder. This process forces the encoder to align its internal feature distributions with a high-quality latent space.
Key Insights
- Generative Features for Recognition: The mapping of an image back into a generative latent space acts as a powerful feature extractor. These "latent codes" and "structural codes" capture the intrinsic geometry of an expression more effectively than standard classification layers.
- Global Context via Transformers: By moving away from CNNs, the model avoids over-relying on single features (e.g., just the mouth for a smile) and instead analyzes the coordinated movement of the entire face.
- Structural Integrity: Ablation studies reveal that "Structure Codes"—representing high-level semantics—are essential for maintaining the geometric "skeleton" of an expression.
Results and Evaluation
The model was evaluated on high-standard datasets including FFHQ and CelebA-HQ for inversion, and RAF-DB, SFEW, and AffectNet for recognition.
ASIT achieved state-of-the-art inversion quality, outperforming models like pSp and e4e in terms of MSE and FID. In the FER domain, IFER reached an accuracy of 87.23% on the RAF-DB dataset, surpassing specialized models such as TransFER. Notably, removing the inversion pre-training stage led to a 4% drop in accuracy, confirming that the generative task is a critical driver of recognition performance.
Conclusion and Future Work
IFER demonstrates that "learning to generate" is a viable path toward "learning to recognize." While the computational cost of Transformers and StyleGAN2 remains a hurdle for mobile deployment, the gain in accuracy and structural understanding is significant. Future research will look toward extending this framework to video-based expression inversion and improving robustness against extreme head poses and occlusions.
Summary updated