Abstract
Generalized Facial Expression Category Discovery (G-FACE) is a challenging task that requires models to recognize known emotions while simultaneously discovering new, complex expression categories in unlabeled data. Traditional methods often struggle with the subtle visual differences and distributional shifts inherent in facial data. This paper introduces DIG-FACE (De-biased G-FACE), a framework that mathematically bounds implicit distributional biases and employs a hierarchical refinement strategy to mitigate explicit visual overlaps. Experimental results on major benchmarks like RAF-DB and AffectNet demonstrate that DIG-FACE significantly outperforms existing state-of-the-art category discovery methods.
The Challenge of Open-World Emotion Recognition
Most facial expression recognition (FER) systems operate in a "closed-set" environment, trained on a fixed list of basic emotions such as happiness, sadness, or surprise. However, real-world human emotion is nuanced and often involves composite expressions (e.g., "happily surprised") that do not fit into predefined categories. Manually labeling every possible complex expression is prohibitively expensive, necessitating Generalized Category Discovery (GCD).
Standard GCD methods designed for distinct objects (like cars vs. dogs) often fail on facial data due to two primary biases:
- Implicit Bias: A distributional gap exists between known and new categories in unlabeled data, which shifts decision boundaries during training.
- Explicit Bias: Facial expressions are visually subtle. The difference between "Anger" and "Disgust" is far smaller than the difference between a "Cat" and a "Bus," leading to heavily overlapped feature spaces.
The DIG-FACE Framework
To address these challenges, DIG-FACE employs a two-pronged debiasing strategy designed to refine the model's understanding of both global distributions and fine-grained facial features.
Implicit Debiasing
The researchers treat the distributional gap as a mathematical optimization problem. They define an F-discrepancy metric to estimate the upper bound of the bias introduced by new categories. By using an adversarial "Min-Max" training process with an auxiliary classification head, the model is forced to minimize this discrepancy, ensuring robust decision boundaries even as new classes are introduced.
Explicit Debiasing
To resolve the problem of overlapping features in subtle expressions, DIG-FACE uses a three-level hierarchical refinement:
- Sample-level: A "Confusing Sample Mining" strategy dynamically increases the weight of hard-to-distinguish samples during training.
- Triplet-level: Contrastive learning—both supervised for known classes and self-supervised for unlabeled data—ensures the model captures fine-grained textures like specific eye or mouth movements.
- Distribution-level: A supervised clustering loss encourages "intra-class compactness" (keeping similar emotions together) and "inter-class separation" (pushing different emotions apart).
Experimental Results
DIG-FACE was evaluated on several major FER benchmarks, including RAF-DB (Basic & Compound), FERPlus, and AffectNet. The model set a new state-of-the-art standard across these datasets.
On the RAF-DB dataset, DIG-FACE improved the accuracy of recognizing known ("Old") classes by 16.2% compared to the previous best method, SimGCD. When tasked with discovering complex compound expressions, it achieved a 20.8% improvement in old category accuracy and a 4.9% boost in new category discovery.
Visualizations of the model's attention maps reveal that DIG-FACE focuses more intensely on key communicative areas—the cheeks, eyes, and mouth corners—than previous baselines. Furthermore, the t-SNE plots above demonstrate that DIG-FACE effectively untangles the overlapped feature spaces that plague standard GCD methods, resulting in distinct, well-separated clusters for both known and discovered expressions.
Conclusion
DIG-FACE demonstrates that by quantifying distributional bias and applying hierarchical feature refinement, models can successfully navigate the "fine-grained" challenges of facial expression discovery. While the current framework assumes a known number of potential clusters, it provides a robust foundation for future "open-world" systems that can incrementally learn and categorize the full spectrum of human emotion without exhaustive manual supervision.
Summary updated