Abstract
FedUFO (Unified Feature learning and Optimization objectives alignment) is a novel framework designed to mitigate the "client drift" phenomenon in Federated Learning (FL) caused by non-IID (not identically distributed) data. By integrating adversarial feature learning with group and global consensus losses, FedUFO ensures that decentralized models remain aligned in both their internal feature representations and their optimization objectives. This approach yields higher overall accuracy and significantly improves fairness across diverse clients.
What Problem Does This Paper Address?
Federated Learning (FL) allows multiple parties (clients) to train a shared model without ever sharing their raw data. However, in real-world scenarios, data is rarely distributed evenly. One client might have only photos of cats, while another has only dogs. This is known as non-IID data.
When data is non-IID, two major issues arise:
- Optimization Inconsistency: Local models are pulled in different directions during training, making the final aggregated global model perform poorly.
- Feature Divergence: Because they see different data, different clients' models learn to "look" at different features, making it hard to create a unified representation.
Existing solutions often ignore these discrepancies or require sharing a small portion of data to "anchor" the models, which compromises privacy. Furthermore, standard FL often results in "unfair" models that work well for some clients but fail for others.
Methods
The authors propose FedUFO, which tackles these problems through a three-step local training process:
- Step 1: Local Training and Exchange: Clients train their local models and exchange them with other clients in the training round.
- Step 2: Information Extraction: Feature expressions and data information are extracted from the global model and other local models to serve as alignment targets.
- Step 3: Posterior Model Training:
- Unified Feature Representation Learning (UAD): The authors introduce an adversarial module where a "discriminator" tries to guess which client a specific feature came from. The feature extractor is trained to "fool" this discriminator, forcing it to learn a generalized feature representation that works across all clients' data distributions.
- Unified Optimization Objective Alignment: To stop models from drifting apart, FedUFO uses two types of consensus:
- Group Consensus Loss (CGR): Clients align their predictions with a weighted average of the predictions from the entire group. This helps the model "see" the broader group distribution without seeing their data.
- Global Consensus Loss (CGL): The local model is also aligned with the global model from the previous iteration, acting as a global anchor to ensure smooth updates.
- Dynamic Sampling Strategy: To improve fairness, the system tracks which clients are performing poorly and samples them more frequently for training rounds, ensuring the global model doesn't "forget" the harder-to-learn data distributions.
Datasets
The researchers evaluated FedUFO across several computer vision benchmarks:
- Classification: MNIST (digits) and CIFAR-10 (objects).
- Re-Identification (Re-ID): VeRI776 (vehicles), MSMT17, and Market1501 (people).
- Fine-Grained Classification: CUB200 (birds).
To simulate the non-IID challenge, they partitioned these datasets so that each client only received a small subset of the total available classes (e.g., in CIFAR-10, each client might only see 2 out of the 10 possible classes).
Key Insights and Ideas
- Alignment on Two Fronts: The core insight is that weight aggregation alone is insufficient. One must align the internal representations (features) and the training targets (objectives) simultaneously to prevent model divergence.
- Consensus as a Proxy for Data Sharing: By using the predictions of other local models as a "consensus" target, FedUFO achieves the benefits of data-sharing methods without violating privacy by sharing raw pixels.
- Fairness is a Choice: The paper demonstrates that "fairness" (reducing the performance gap between the best and worst clients) doesn't have to come at the cost of overall accuracy. By aligning objectives, the "worst" clients improve significantly, which pulls the global average up.
Key Results
FedUFO consistently outperformed state-of-the-art FL methods like FedAVG, FedMeta, and FML.
- Accuracy Boost: On the challenging CIFAR-10 non-IID task, FedUFO achieved 67.45% accuracy, compared to FedAVG’s 59.95%.
- Superior Fairness: The standard deviation of performance across clients ($P_{std}$) dropped dramatically. On CIFAR-10, it fell from 9.23 (FedAVG) to 3.54 (FedUFO), meaning performance was much more consistent across different users.
- Worst-Case Improvement: The performance of the "worst" client ($P_w$) on CIFAR-10 jumped from 32.50% to 55.40%, a massive gain for users with outlier data.
Limitations and Future Work
- Communication Overhead: The Group Consensus (CGR) and Adversarial (UAD) modules require clients to exchange local models. This increases the amount of data sent over the network compared to simple FedAVG.
- Cross-Silo vs. Cross-Device: Due to storage and communication requirements, FedUFO is currently best suited for "cross-silo" FL (e.g., 2–100 hospitals or banks) rather than "cross-device" FL (e.g., millions of mobile phones).
- Future Directions: The authors plan to explore more sophisticated ways to align group and global objectives that might require less communication overhead.
Summary updated