FedAPA: Server-side Gradient-Based Adaptive Personalized Aggregation for Federated Learning on Heterogeneous Data

Yuxia Sun, Aoxiang Sun, Siyi Pan, Zhixiao Fu, Jingcai Guo

Abstract

Personalized federated learning (PFL) tailors models to clients' unique data distributions while preserving privacy. However, existing aggregation-weight-based PFL methods often struggle with heterogeneous data, facing challenges in accuracy, computational efficiency, and communication overhead. We propose FedAPA, a novel PFL method featuring a server-side, gradient-based adaptive aggregation strategy to generate personalized models, by updating aggregation weights based on gradients of client-parameter changes with respect to the aggregation weights in a centralized manner. FedAPA guarantees theoretical convergence and achieves superior accuracy and computational efficiency compared to 10 PFL competitors across three datasets, with competitive communication overhead.

Introduction

Standard Federated Learning (FL), exemplified by the FedAvg algorithm, assumes data is Independent and Identically Distributed (IID). However, real-world data is typically non-IID, exhibiting statistical heterogeneity that leads to "client drift." In such scenarios, a single global model often fails to perform optimally for any individual participant.

Personalized Federated Learning (PFL) addresses this by providing each client with a tailored model. Despite its promise, many existing PFL methods suffer from three primary drawbacks:

  1. Static Metrics: Reliance on fixed rules (e.g., parameter similarity) that do not adapt to training progress.
  2. High Complexity: Use of heavy "Hypernetworks" or complex local computations that are unsuitable for resource-constrained mobile devices.
  3. Communication Overhead: Excessive data exchange requirements that strain limited bandwidth.

Methodology: Adaptive Personalized Aggregation

The proposed FedAPA (Adaptive Personalized Aggregation) framework shifts the complexity of personalization from the client to the server, ensuring efficiency for edge devices.

Overview of the FedAPA architecture and its 7-step server-side adaptive aggregation process.

Key Components

  • Adaptive Personalized Aggregation (APA): The server maintains a weight vector for every client, determining how much knowledge each client should "borrow" from others.
  • Gradient-Based Weight Updates: FedAPA uses the client-parameter delta (the difference between received and returned models) as a proxy for local loss. The server applies gradient descent to update aggregation weights, effectively learning optimal neighbor contributions.
  • Partial Model Sharing (PMS): To balance performance and bandwidth, FedAPA shares only the "feature extractor" layers while keeping the "classifier head" strictly local.
  • Server-Side Post-Processing: Stability is maintained through three steps: Clipping (preventing extreme weight swings), Self-weight adjusting (ensuring a minimum 50% trust in local data), and Normalization.

Experimental Evaluation

FedAPA was evaluated against 10 state-of-the-art competitors (e.g., FedALA, pFedHN, FedPAC) using three image classification benchmarks: FMNIST (simple), CIFAR-10 (medium), and CIFAR-100 (complex). Tests were conducted under both Pathological and Practical Non-IID settings with varying client counts (20 and 50).

Results and Discussion

FedAPA demonstrated superior performance across nearly all tested scenarios, achieving state-of-the-art accuracy with minimal overhead.

Performance comparison of FedAPA against 10 state-of-the-art PFL methods across various datasets and client configurations.

Key Findings

  • Superior Accuracy: FedAPA outperformed competitors in 10 out of 12 experimental scenarios. Notably, on CIFAR-100 with 50 clients, it achieved 75.09% accuracy, a massive improvement over FedAvg's 11.84%.
  • Efficiency: The method is computationally efficient, adding only 0.34 seconds per iteration compared to FedAvg. It also maintains low communication costs through Partial Model Sharing.
  • Fast Convergence: Stable accuracy is typically reached within 10 rounds, significantly faster than many peer algorithms.
  • The 50% Anchor: Fixing the "self-weight" at 0.5 proved to be a robust heuristic, ensuring local data characteristics are prioritized while still benefiting from collaborative knowledge.

Conclusion

FedAPA introduces an elegant, server-side optimization strategy that treats model parameter changes as a surrogate for local loss. This allows for effective personalization without increasing client-side burden or compromising privacy. While highly effective for vision tasks with moderate client pools, future research will focus on scaling to thousands of clients and evaluating robustness against advanced gradient-based privacy attacks.

Summary updated

Abstract

Personalized federated learning (PFL) tailors models to clients' unique data distributions while preserving privacy. However, existing aggregation-weight-based PFL methods often struggle with heterogeneous data, facing challenges in accuracy, computational efficiency, and communication overhead. We propose FedAPA, a novel PFL method featuring a server-side, gradient-based adaptive aggregation strategy to generate personalized models, by updating aggregation weights based on gradients of client-parameter changes with respect to the aggregation weights in a centralized manner. FedAPA guarantees theoretical convergence and achieves superior accuracy and computational efficiency compared to 10 PFL competitors across three datasets, with competitive communication overhead.
BibTeX
@inproceedings{sun2025fedapa,
  author = {Yuxia Sun and Aoxiang Sun and Siyi Pan and Zhixiao Fu and Jingcai Guo},
  title = {{FedAPA: Server-side Gradient-Based Adaptive Personalized Aggregation for Federated Learning on Heterogeneous Data}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.24963/ijcai.2025/692},
}
Figure