Introduction
Facial Expression Recognition (FER) in real-world scenarios is a complex challenge for computer vision. While deep learning has significantly advanced the field, these models typically require vast amounts of accurately labeled data. In "in-the-wild" datasets, semi-supervised learning (SSL) faces two primary hurdles: label ambiguity and the underutilization of unlabeled data.
Human emotions are inherently subjective; a labeler might categorize a face as "Surprise," while others see "Fear." Standard SSL methods often treat initial labels as absolute ground truth, which can propagate errors. Furthermore, most SSL techniques, such as pseudo-labeling, only utilize unlabeled images when the model is highly confident (e.g., >95%). This results in a significant loss of information from "difficult" or low-confidence images that are typically discarded.
The LION Framework
The authors propose LION (Label DIsambiguation and PrOgressive Negative Learning) to address these issues by cleaning labeled data and extracting insights from uncertain unlabeled samples.
Label DIsambiguation (LD) Module
The LD module audits the labeled dataset to distinguish between "clear" and "ambiguous" data. Clear data—where the model's prediction aligns with the provided label with high confidence—is used to calculate "prototypes," which serve as feature-space anchors for each emotion.
Ambiguous images are assigned a candidate label set (a multi-hot vector). By leveraging the established prototypes and a moving-average weight strategy, the model iteratively "re-labels" these images toward their most likely latent truth, effectively correcting the noise in the original supervision.
PrOgressive Negative Learning (PNL) Module
For unlabeled images where the model remains uncertain, LION employs Negative Learning. Rather than forcing a potentially incorrect "positive" label (e.g., "This is Happy"), the PNL module identifies "complementary" labels (e.g., "This is definitely NOT Disgust").
Through iterative mining, the model progressively rules out classes with the lowest probabilities. To ensure robustness, a Negative Consistency Loss (NC-Loss) penalizes the model if its negative predictions for different augmented views of the same image are inconsistent. This allows the model to learn from low-confidence data without the risk of introducing incorrect pseudo-labels.
Experimental Results
LION was evaluated on three benchmark datasets: RAF-DB, AffectNet, and SFEW. The framework achieved state-of-the-art (SOTA) results, demonstrating particular strength in low-data regimes:
- Minimal Data Performance: On RAF-DB with only 100 labeled images, LION achieved 67.83% accuracy, outperforming the previous best method (Ada-CM) by 5.47%.
- Challenging Environments: On the difficult SFEW dataset (400 labels), LION reached 54.18%, a 3.73% improvement over the runner-up.
- Baseline Comparison: In several instances, LION's semi-supervised approach outperformed fully-supervised baselines, suggesting that its method of handling ambiguity is more effective than simply increasing the volume of potentially noisy labels.
Conclusion and Future Work
The core insight of LION is the "Power of No." In emotion recognition, boundaries between similar expressions like "Anger" and "Disgust" are often blurry, but it is much easier for a model to determine what an expression is not. By combining dynamic relabeling with progressive negative learning, LION creates a mutually reinforcing system where better features lead to more accurate prototypes and vice versa.
Future research may address the computational complexity introduced by the iterative negative learning loops and the sensitivity of the model to its various thresholds (τ, μ, γ, δ). However, the success of LION suggests that this framework could be broadly applicable to other domains characterized by high label noise and ambiguity.
Summary updated