PA3Fed: Period-Aware Adaptive Aggregation for Improved Federated Learning

Chengxiang Huang, Bingyan LiuAAAI 2025 · DOI

TL;DR

This paper introduces PA³Fed, a novel federated learning approach that adaptively adjusts client contributions to the global model based on their performance during "Critical Learning Periods" (CLPs). By focusing on these crucial early training phases, PA³Fed significantly improves model accuracy and stability, especially in challenging non-i.i.d. data scenarios, without disrupting existing inter-client optimization methods.

What Problem Does This Paper Address?

Federated Learning (FL) enables collaborative model training across decentralized devices while preserving data privacy. However, a major hurdle is "statistical heterogeneity," where client data distributions vary significantly (non-i.i.d.). This heterogeneity can lead to unstable training, causing the global model to diverge from optimal performance. Existing FL methods often focus on optimizing interactions between clients, sometimes overlooking valuable information within a single client's training process. Specifically, the paper highlights that the loss curves of individual clients can be highly volatile, especially in the early stages of training. This volatility, if not properly managed, can compromise the overall stability and effectiveness of the FL system. The authors aim to leverage this intra-client volatility to improve aggregation, particularly during periods of high information content.

Methods

The core of PA³Fed lies in its "Period-Aware Adaptive Aggregation." Instead of a uniform aggregation strategy, PA³Fed dynamically adjusts how much each client's model contributes to the global model. This is achieved through two key steps:

  1. Period Check: The system identifies "Critical Learning Periods" (CLPs). These are periods in training where the model's learning is most sensitive and information-rich. The paper uses Federated Gradient Norm (FGN) as a real-time proxy for the Fisher Information Matrix (FIM) to detect these periods. FGN is computationally less expensive than FIM, making it suitable for online detection. If the system is within a CLP, adaptive aggregation is activated; otherwise, standard aggregation (like FedAvg) is used.
  2. Adaptive Aggregation: During CLPs, PA³Fed calculates an adaptive aggregation coefficient for each client. This coefficient is based on the client's performance variations (specifically, the change in a performance score derived from loss and accuracy) between consecutive training rounds. Clients showing more stable performance (less volatility) receive a higher aggregation weight, while those with higher volatility receive a lower weight. This is mathematically formulated using an exponential function that adjusts the client's contribution based on the difference in their performance scores. This intra-client focus is designed to be orthogonal to existing inter-client optimization methods, allowing for easy integration.

Datasets

The paper evaluates PA³Fed on three standard image classification datasets:

  • CIFAR-10
  • CIFAR-100
  • Fashion-MNIST

These datasets are used with varying degrees of non-i.i.d. settings, controlled by a Dirichlet distribution parameter 'h', where lower 'h' values indicate higher heterogeneity.

Key Insights and Ideas

  • Critical Learning Periods (CLPs) are crucial: The paper reinforces the idea that early training phases (CLPs) hold disproportionately high information content and are critical for overall model performance and stability.
  • Intra-client information is valuable: Moving beyond solely inter-client optimization, PA³Fed demonstrates that analyzing a client's own training trajectory (loss fluctuations, performance score changes) can provide rich signals for improving global aggregation.
  • FGN as a practical CLP detector: Using Federated Gradient Norm (FGN) as a real-time indicator for CLPs is a practical innovation, overcoming the computational burden of traditional FIM calculations.
  • Adaptive aggregation can be period-specific: Applying adaptive aggregation only during CLPs, rather than throughout the entire training, strikes a balance between leveraging informative periods and avoiding potential information loss from overly stable clients in later stages.

Key Results

  • Significant Accuracy Improvement: PA³Fed consistently improves test accuracy across various baseline methods (FedAvg, FedProx, FedNova, VRL-SGD, FedMut) and datasets. Notably, it achieves an approximate 15% improvement on CIFAR-10 under high non-i.i.d. conditions (h=0.1).
  • Enhanced Stability: Table 1 shows that PA³Fed demonstrates superior stability across metrics like Standard Deviation, Moving Average Standard Deviation, Mean Absolute Deviation, Range, and Maximum Change Rate compared to FedAvg. Figure 5 further illustrates this by showing lower loss standard deviation for PA³Fed, both during CLPs and rest periods.
  • Faster Convergence: Figure 4 indicates that models augmented with PA³Fed converge more rapidly on CIFAR-10.
  • Robustness to Non-IID: Figure 8 shows that PA³Fed yields greater performance improvements with higher non-i.i.d. degrees, while still performing well even as heterogeneity decreases.

Limitations and Future Work

  • Hyperparameter Tuning: The paper acknowledges the need to tune hyperparameters like β (controlling adaptation degree) and δ (for CLP detection sensitivity). While they found β=0.3 to be optimal in experiments, its generalizability might require further investigation.
  • Computational Overhead: While FGN is more efficient than FIM, the adaptive aggregation still introduces some computational overhead on the server side.
  • Future Work: The authors suggest exploring more sophisticated methods for CLP detection and adaptive aggregation coefficient calculation. They also hint at the potential for PA³Fed to be integrated with other advanced FL techniques for further performance gains.

Summary updated

BibTeX
@inproceedings{huang2025pa,
  author = {Chengxiang Huang and Bingyan Liu},
  title = {{PA3Fed: Period-Aware Adaptive Aggregation for Improved Federated Learning}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i16.33912},
}
Figure