Abstract
Federated learning (FL) allows agents to jointly train a global model without sharing their local data. However, due to the heterogeneous nature of local data, it is challenging to optimize or even define fairness of the trained global model for the agents. For instance, existing work usually considers accuracy equity as fairness for different agents in FL, which is limited, especially under the heterogeneous setting, since it is intuitively "unfair" to enforce agents with high-quality data to achieve similar accuracy to those who contribute low-quality data, which may discourage the agents from participating in FL. In this work, we propose a formal FL fairness definition, fairness via agent-awareness (FAA), which takes different contributions of heterogeneous agents into account. Under FAA, the performance of agents with high-quality data will not be sacrificed just due to the existence of large amounts of agents with low-quality data. In addition, we propose a fair FL training algorithm based on agent clustering (FOCUS) to achieve fairness in FL measured by FAA. Theoretically, we prove the convergence and optimality of FOCUS under mild conditions for linear and general convex loss functions with bounded smoothness. We also prove that FOCUS always achieves higher fairness in terms of FAA compared with standard FedAvg under both linear and general convex loss functions. Empirically, we show that on four FL datasets, including synthetic data, images, and texts, FOCUS achieves significantly higher fairness in terms of FAA while maintaining competitive prediction accuracy compared with FedAvg and state-of-the-art fair FL algorithms.
Introduction
Federated Learning (FL) has emerged as a powerful paradigm for privacy-preserving collaborative machine learning. However, the practical deployment of FL is often hindered by data heterogeneity. Agents—ranging from mobile devices to medical institutions—contribute data with varying distributions, noise levels, and quality.
Traditional fairness metrics in FL, such as accuracy parity, aim to equalize performance across all participating agents. This paper argues that such an approach is fundamentally flawed in heterogeneous environments. Forcing a high-quality data contributor to accept the same accuracy as a noisy "outlier" agent effectively imposes a "tax" on quality, discouraging participation from the most valuable agents.
Fairness via Agent-Awareness (FAA)
To address this, the authors propose Fairness via Agent-Awareness (FAA). Instead of raw accuracy, FAA focuses on excess risk: the difference between an agent's current test loss and their Bayes optimal error (the minimum possible error achievable on their specific data distribution). Fairness is achieved when the gap in excess risk between any two agents is minimized. This "equity of gains" ensures that every agent benefits from the collaboration relative to the inherent potential of their own data.
The FOCUS Algorithm
To optimize for FAA, the authors developed FOCUS (Fairness via Agent-Clustering). FOCUS employs a soft-clustering approach within an Expectation-Maximization (EM) framework to group agents with similar data distributions:
- E-step (Soft Assignment): Agents evaluate their local data against $M$ cluster models and update their "soft" membership probabilities.
- M-step (Weighted Aggregation): The server updates the $M$ cluster models by performing a weighted average of agent updates, where weights are determined by the soft assignments.
- Inference: Agents make predictions using an ensemble of all cluster models, weighted by their membership probabilities.
This soft clustering allows for knowledge transfer between related groups while isolating outliers that might otherwise degrade the global model.
Experimental Results
The effectiveness of FOCUS was evaluated on synthetic data, Rotated MNIST, Rotated CIFAR-10, and Yelp/IMDb sentiment analysis.
Fairness and Robustness
FOCUS consistently achieves lower FAA scores (indicating higher fairness) than standard FedAvg. As shown in the figure below, FOCUS (green bars) maintains a test loss much closer to the Bayes optimal surrogate (orange bars) across different client groups (C1, C2, C3) compared to FedAvg (blue bars).
Convergence and Accuracy
Beyond fairness, FOCUS demonstrates superior efficiency. In the Rotated MNIST task, FOCUS reached 90% accuracy in just 29 rounds, whereas FedAvg required 177 rounds. Other fair FL algorithms, such as q-FFL and AFL, often failed to reach high accuracy thresholds or converged much more slowly.
Conclusion
FOCUS introduces a paradigm shift in federated fairness by accounting for agent-specific data quality. By optimizing for "equity of gains" through soft clustering, FOCUS provides a robust, fair, and highly efficient framework for collaborative learning on heterogeneous data.
Summary updated