Introduction
Standard Federated Learning (FL) aims to train a single global model across decentralized clients. However, real-world data is often "non-IID" (not identically and independently distributed), leading to client drift where a one-size-fits-all model performs poorly for individual participants. Personalized Federated Learning (PFL) addresses this by tailoring models to local data distributions. This paper introduces FedAPA (Adaptive Personalized Aggregation), a server-side framework that efficiently learns optimal aggregation weights to maximize personalization without the heavy computational or communication costs typical of existing PFL methods.
The FedAPA Framework
The core innovation of FedAPA is shifting the complexity of learning personalization weights from the client to the server. Most PFL methods rely on static metrics or expensive client-side computations. FedAPA, instead, uses a gradient-based approach on the server.
1. Server-Side Weighting and Proxy Loss
The server maintains an aggregation-weight vector $A_i$ for every client $i$. To create a personalized model $\bar{\theta}_i$, the server computes a weighted sum of all clients' parameters. The challenge is updating these weights without direct access to local data. FedAPA solves this using the "Proxy Loss" Trick: the server observes the difference ($\Delta\theta_i$) between the model sent to a client and the updated model received back. The authors mathematically demonstrate that half the squared norm of this delta serves as an effective proxy for the local loss gradient, allowing the server to perform gradient descent on the weights $A_i$.
2. Partial Model Sharing and Stability
To balance collaboration and privacy, FedAPA employs Partial Model Sharing (PMS). Clients share their "feature extractors" (layers responsible for data representation) while keeping their "classifiers" (final decision layers) private. Furthermore, the server applies post-processing to the weights, including clipping and a "self-weighting" rule that ensures a client's own parameters always carry significant weight (typically fixed at 0.5) to prevent local knowledge from being overwhelmed by the collective.
Experimental Evaluation
The authors evaluated FedAPA against 10 state-of-the-art competitors (e.g., FedALA, pFedHN, FedPAC) using three image classification benchmarks: FMNIST, CIFAR-10, and CIFAR-100. These were tested under both "Pathological" and "Practical" non-IID scenarios.
Performance and Convergence
FedAPA consistently achieved superior accuracy, particularly in practical non-IID settings which better reflect real-world data imbalances. As shown in the convergence plots, FedAPA not only reaches higher final accuracy but also stabilizes faster than many complex alternatives.
Efficiency and Overhead
A major advantage of FedAPA is its efficiency. While methods like FedPAC can triple the time per iteration compared to standard FedAvg, FedAPA maintains a computational footprint nearly identical to the baseline. For instance, on CIFAR-10, FedAPA required approximately 18 seconds per iteration, compared to 17 seconds for FedAvg and 64 seconds for FedPAC. This "near-zero overhead" makes it highly viable for resource-constrained environments.
Conclusion
FedAPA represents a significant step forward in PFL by centralizing the optimization of personalization weights. By leveraging parameter deltas as a proxy for local loss, it achieves state-of-the-art performance with minimal computational and communication costs. Future work may address the $O(M^2)$ storage requirement for weight vectors in massive-scale deployments and further investigate the differential privacy implications of server-side weight updates.
Summary updated