Standard Federated Learning (FL) aims to train a single global model across decentralized devices. This works beautifully when every client’s data looks roughly the same. However, in the real world, data is rarely "Independent and Identically Distributed" (IID). A hospital in Tokyo has different patient demographics than one in New York; a smartphone user in London types differently than one in Seoul.
When data is heterogeneous (non-IID), the "average" global model produced by algorithms like FedAvg often performs poorly for everyone. It suffers from model drift, where local updates pull the model in conflicting directions, resulting in a global consensus that represents no one well. Personalized Federated Learning (PFL) shifts the goal: instead of one model to rule them all, we use the collective power of the network to help each client train a model tailored to their specific needs.
Decoupling Representation from Classification
One of the most intuitive ways to personalize a model is to recognize that neural networks are generally composed of two parts: a feature extractor (the body) that learns to see patterns, and a classifier (the head) that interprets those patterns into labels.
In heterogeneous settings, the "meaning" of features might be universal, but the labels are local. Lixuan Yang and Dario Rossi proposed HDAFL (Heterogeneous Data-Aware Federated Learning), which shares only the feature-extraction layers globally while keeping classification layers private. This allows the network to build a powerful, shared "visual vocabulary" while letting each client decide how to categorize those visuals.
Jaehoon Oh and Se-Young Yun took this further with FedBABU (Towards Enhanced Representation for Federated Image Classification). They found that by freezing the classifier head during global training, they could force the global model to focus entirely on learning a robust, general-purpose feature extractor. Once the global "body" is trained, clients simply unfreeze the head and fine-tune it on their local data, achieving rapid and highly effective personalization.
The Global Model as a Prior
Rather than splitting the model, another approach treats the global model as a "guide" or a "prior." In this view, the local model shouldn't be exactly the same as the global one, but it shouldn't stray too far either.
Mingjia Shi and Jiancheng Lv introduced pFedBreD (Personalized Federated Learning with Hidden Information on Personalized Prior), which frames personalization as a Bayesian inference problem. The global model acts as a prior distribution, and local training uses Bregman divergence to ensure the personalized model stays within a reasonable "distance" of the collective knowledge. Similarly, FedU uses Laplacian regularization to manage the relationships between different local tasks, essentially treating the federated network as a multi-task learning problem where the goal is to minimize the loss across all unique client distributions simultaneously.
Selective Aggregation and Parameter Efficiency
As models grow larger, sharing entire sets of weights becomes a communication bottleneck. Researchers have begun looking at Parameter-Efficient Fine-Tuning (PEFT) as a vehicle for personalization.
A striking example is the use of Low-Rank Adaptation (LoRA). Pengxin Guo and Liangqiong Qu discovered an "asymmetry" in how LoRA matrices behave in federated settings (Selective Aggregation for Low-Rank Adaptation in Federated Learning). They found that sharing the $A$ matrix (which captures the low-rank structure of the data) globally while keeping the $B$ matrix (which maps that structure to specific outputs) local provides a powerful balance of global coordination and local specialization.
This philosophy of "decomposing" weights is also seen in FedWeIT (Federated Continual Learning with Weighted Inter-client Transfer). Jaehong Yoon and colleagues decomposed neural weights into base global weights and sparse, task-specific local weights. This prevents "inter-client interference"—where one client's update overwrites another's progress—allowing for continuous learning across diverse tasks.
Beyond Weights: Prototypes and Distillation
Sometimes, the best way to share knowledge isn't to share weights at all, but to share "concepts."
ProtoFed (Prototype Helps Federated Learning) replaces the final biased classifier with class prototypes—mean feature representations of each class. Instead of calculating a probability via a weight matrix, the model performs inference by measuring the distance between a sample's features and these shared prototypes. This bypasses the "label skew" problem entirely because the prototypes provide a stable, unbiased reference point for what a "cat" or a "dog" looks like across the entire network.
Knowledge distillation offers another path. Chuhan Wu and Lingjuan Lyu developed FedKD (Communication-efficient federated learning via knowledge distillation), which uses a mentor-mentee setup. Clients train a large "mentor" model locally but only communicate the predictions (or a smaller "mentee" model) to the server. This allows for heterogeneous model architectures—different clients can use different sized models—while still benefiting from a shared "teacher" signal.
Theoretical Foundations
The move toward personalization isn't just an empirical trend; it is supported by new theoretical frameworks. Filip Hanzely and Peter Richtárik established the first lower bounds for communication in PFL (Lower Bounds and Optimal Algorithms for Personalized Federated Learning), proving why local methods like FedProx are mathematically optimal for highly heterogeneous data.
Furthermore, Gary Cheng and Karan N. Chadha provided an asymptotic analysis (Federated Asymptotics) showing that in many high-dimensional regimes, simple local fine-tuning of a global model is surprisingly competitive with more complex meta-learning algorithms. This suggests that the most effective path to personalization often lies in finding the right balance between global initialization and local adaptation.