Abstract
Facial Expression Recognition (FER) in real-world scenarios is often hampered by ambiguous expressions, low-quality images, and noisy labels. While Federated Learning (FL) offers a privacy-preserving solution, standard approaches struggle with heterogeneous sample uncertainty across different clients. We propose AMY (uncertainty-Aware label refineMent on hYpergraphs), a framework that leverages hypergraph neural networks to capture high-order relationships between facial images. AMY identifies unreliable samples and refines their labels locally using hypergraph label propagation. Experimental results on RAF-DB and FERPlus demonstrate that AMY significantly outperforms state-of-the-art decentralized models, particularly in highly heterogeneous data environments.
Introduction
Facial Expression Recognition is a complex task due to the inherent subjectivity of human emotions and the prevalence of noisy data. Traditional centralized training poses significant privacy risks regarding sensitive facial data. Federated Learning (FL) addresses this by training models locally on user devices. However, a significant "gap" remains: heterogeneous sample uncertainty. In a federated network, data quality varies wildly between clients—one may have clear, well-labeled images, while another has blurry photos or inconsistent labeling. Existing methods often rely on pairwise image relationships, failing to capture the complex, high-order clusters of expressions found in real-world datasets.
Methodology: The AMY Framework
The AMY framework introduces a personalized federated learning setup where each client maintains a local model designed to handle uncertainty through three primary components:
- Uncertainty Estimation (UE) Block: This block employs a Hypergraph Neural Network (HGNN). Unlike standard graphs that model pairwise connections, hypergraphs use "hyperedges" to connect multiple vertices, allowing the model to capture high-order relationships and more effectively identify outlier (uncertain) samples.
- Personalized Uncertainty Estimator (PUE): A local module that assigns uncertainty weights to each image. The PUE remains private to the client, allowing it to learn specific noise patterns unique to that user's data without compromising privacy.
- Expression Classification (EC) Block: For samples identified as uncertain, the model utilizes Hypergraph Label Propagation (HLPN). By examining "neighboring" clusters in the hypergraph, the model can self-correct noisy labels before the final classification step.
To ensure global alignment without sharing raw data, the server aggregates class prototypes (feature centroids) using the FedProto algorithm, rather than raw model weights.
Experimental Setup
The researchers evaluated AMY on two major benchmarks: RAF-DB (~30,000 images, 7 categories) and FERPlus (8 categories with crowd-sourced labels). To simulate a realistic federated environment, the data was partitioned among 10 clients using a Dirichlet distribution ($\alpha$), allowing for controlled levels of data heterogeneity.
Results and Discussion
AMY consistently outperformed traditional FL methods (e.g., FedAvg) and specialized FER uncertainty-suppression methods like SCN and RUL.
Key Findings:
- Accuracy Improvements: On the RAF-DB dataset with high heterogeneity ($\alpha=0.1$), AMY achieved 90.97% accuracy, significantly higher than the baseline (87.07%) and FedAvg (86.10%).
- Robustness to Skewed Data: AMY maintained superior performance even when data distributions were highly unbalanced across clients, proving the effectiveness of prototype-based collaboration.
- Qualitative Validation: Visualization confirmed that AMY correctly assigns high uncertainty to images with occlusions or extreme lighting, which typically degrade standard FER performance.
Conclusion
The AMY framework demonstrates that modeling high-order relationships via hypergraphs is a powerful tool for identifying and refining uncertain data in federated FER. By keeping uncertainty estimation local and personalized, the system turns data heterogeneity into a manageable characteristic rather than a performance bottleneck. Future work will explore making uncertainty thresholds learnable and optimizing the computational efficiency of hypergraph operations for edge devices.
Summary updated