On Bridging Generic and Personalized Federated Learning

Hong-You Chen, Wei-Lun Chao

Abstract

Federated learning (FL) collaboratively trains models across multiple clients without direct data access. However, data heterogeneity often forces a trade-off between generic performance (a global model for all) and personalized performance (specific models for each client). We propose Federated Robust Decoupling (Fed-RoD), a framework that bridges this gap by explicitly decoupling a model's dual duties. By using a shared feature extractor with two specialized prediction heads—one for generic tasks using balanced risk minimization and one for personalized tasks using empirical risk—Fed-RoD achieves state-of-the-art results in both domains simultaneously.

The Dilemma of Data Heterogeneity

In Federated Learning, clients often possess non-IID (non-identically and independently distributed) data. This heterogeneity creates a fundamental tension: should the server optimize for a single global model that generalizes well to new users (Generic FL), or should it prioritize models tailored to each client's specific data distribution (Personalized FL)?

Historically, these have been treated as separate objectives. Generic FL algorithms like FedAvg often struggle with local client performance, while personalized approaches may fail to produce a robust global model. Interestingly, the authors observe that standard generic algorithms actually generate strong personalized models during local training, but these are typically discarded in favor of the global average at the end of each round.

Performance gap between global weights (w-bar) and local weights (w_m) in generic and personalized accuracy.

Federated Robust Decoupling (Fed-RoD)

To resolve this conflict, Fed-RoD introduces a "two-loss, two-predictor" framework. The architecture consists of a shared feature extractor and two distinct prediction heads:

  1. Generic Head ($h^G$): This head is trained using Balanced Risk Minimization (BRM). By employing a Balanced Softmax loss, it learns a representation that treats all classes fairly, regardless of local label skew. This head is aggregated at the server to form the global model.
  2. Personalized Head ($h^P$): This lightweight module is trained to minimize the client's specific Empirical Risk. It adapts the generic representation to the local data distribution.
Evolution of FL architectures from standard ERM/BRM to the proposed Fed-RoD variants.

The framework offers two implementation paths for the personalized predictor:

  • Fed-RoD (linear): Uses a simple local linear layer for personalization.
  • Fed-RoD (hyper): Utilizes a hypernetwork that generates personalized weights based on the client's class distribution. This enables "zero-shot" personalization for new clients who were not part of the training set.

Key Insights and Results

  • Implicit Regularization: The process of re-initializing local models with the global average acts as a powerful regularizer, preventing overfitting to small local datasets more effectively than standard $L_2$ penalties.
  • Decoupling Benefits: By separating the "generic" duty (balanced representation) from the "personalized" duty (local bias adaptation), Fed-RoD prevents objective competition, allowing both to reach peak performance.
  • State-of-the-Art Performance: Evaluated on CIFAR-10, CIFAR-100, and EMNIST, Fed-RoD consistently outperformed both generic (FedAvg, FedProx) and personalized (Ditto, pFedMe) baselines. On CIFAR-10 with high skew, Fed-RoD achieved 76.9% generic accuracy (vs. FedAvg's 68.6%) and reached personalized scores as high as 94%.

Conclusion

Fed-RoD demonstrates that generic and personalized performance are not mutually exclusive. By decoupling these tasks into specialized heads on a shared backbone, federated systems can provide both a robust global model for new users and highly accurate personalized models for existing ones, effectively bridging the two major tracks of federated learning research.

Summary updated

Abstract

Federated learning is promising for its capability to collaboratively train models with multiple clients without accessing their data, but vulnerable when clients' data distributions diverge from each other. This divergence further leads to a dilemma: "Should we prioritize the learned model's generic performance (for future use at the server) or its personalized performance (for each client)?" These two, seemingly competing goals have divided the community to focus on one or the other, yet in this paper we show that it is possible to approach both at the same time. Concretely, we propose a novel federated learning framework that explicitly decouples a model's dual duties with two prediction tasks. On the one hand, we introduce a family of losses that are robust to non-identical class distributions, enabling clients to train a generic predictor with a consistent objective across them. On the other hand, we formulate the personalized predictor as a lightweight adaptive module that is learned to minimize each client's empirical risk on top of the generic predictor. With this two-loss, two-predictor framework which we name Federated Robust Decoupling (Fed-RoD), the learned model can simultaneously achieve state-of-the-art generic and personalized performance, essentially bridging the two tasks.
BibTeX
@inproceedings{chen2022bridging,
  author = {Hong-You Chen and Wei-Lun Chao},
  title = {{On Bridging Generic and Personalized Federated Learning}},
  booktitle = {International Conference on Learning Representations},
  year = {2022},
  url = {https://openreview.net/forum?id=QuBYtHLGho},
}
Figure