TL;DR: FAIR-FER introduces a novel approach to reducing demographic bias in Facial Expression Recognition (FER) by aligning the latent spaces of different groups. By using a shared-weight Variational Autoencoder (VAE) and an adversarial discriminator, the model learns to extract emotion-related features while intentionally ignoring protected attributes like age, gender, and race.
Introduction
Facial Expression Recognition (FER) systems are increasingly integrated into various technologies, yet they often suffer from demographic biases inherited from human-annotated datasets. For instance, studies have shown that human labelers frequently associate younger faces with positive emotions (like happiness) and older faces with negative ones (like sadness or disgust). When deep learning models train on such data, they learn "shortcuts"—predicting emotions based on demographic features rather than actual facial muscle movements. This leads to a lack of "equal-odds" fairness, where accuracy varies significantly across different groups. FAIR-FER addresses this by ensuring the model's internal representation of a face is "blind" to protected attributes like age, gender, and race.
Methodology: Attribute Disentanglement
The authors propose a two-part architecture designed for attribute disentanglement. The core idea is to align the latent representations of different demographic groups so that they are indistinguishable to downstream classifiers.
The system consists of several key components:
- Shared-Weight VAE: A Variational Autoencoder encodes images into a latent space. The encoder's weights are shared across all demographic groups to promote a unified representation. A generator is used during training to ensure the latent space retains structural facial information through reconstruction loss.
- Adversarial Alignment: An adversarial discriminator attempts to predict protected attributes (e.g., age, race) from the latent code. The encoder is trained to "fool" this discriminator, effectively scrubbing demographic-specific information from the representation. The training objective balances KL-Divergence, discriminator loss, and style-reconstruction loss.
- MBConv Classifier: The resulting "fair" latent representation is fed into a classification module using Mobile Inverted Bottleneck Convolution (MBConv) blocks to predict one of seven basic emotions.
Experimental Results
The model was evaluated on the RAF-DB (Real-world Affective Faces Database), which contains 12,271 training images and 3,068 test images annotated with seven basic emotions and demographic metadata. The results demonstrate that FAIR-FER not only improves fairness but also enhances overall predictive accuracy.
As shown in the results, FAIR-FER achieved an overall accuracy of 76.1%, surpassing the baseline of 74.8%. More importantly, it significantly improved fairness metrics for Race (from 91.6 to 94.2) and Age (from 82.1 to 84.8). By forcing the model to ignore demographic "noise," it is compelled to focus more effectively on the actual visual cues relevant to facial expressions, such as muscle movements. This effectively counteracts the "happy adolescent" bias where models over-predict positive emotions for younger faces.
Conclusion and Future Work
FAIR-FER provides a robust proof-of-concept for using latent alignment to shield FER models from societal biases. While currently tested on basic emotions, future work will explore its scalability to complex "in-the-wild" datasets and its generalizability to other domains like medical imaging. Although the adversarial VAE pipeline introduces additional computational overhead, the gains in both fairness and accuracy make it a promising direction for ethical AI development.
Summary updated