Age-Aware Partial Gradient Update Strategy for Federated Learning Over the Air

Ruihao Du, Zeshen Li, Howard H. YangarXiv 2025 · DOI · arXiv

Abstract

This paper introduces AgeTop-k, a novel gradient compression strategy designed for Over-the-Air Federated Learning (OTA-FL). By integrating gradient magnitude with the "Age of Information" (AoI), the strategy ensures that model updates are both numerically significant and timely. This approach prevents specific parameters from becoming "stale," a common failure mode in standard sparsification techniques, thereby accelerating convergence and improving final model accuracy.

Introduction

In Federated Learning (FL), a central server aggregates updates from multiple clients to train a global model. To enhance scalability, Over-the-Air (OTA) computation leverages the superposition property of wireless signals, allowing simultaneous update aggregation. However, OTA-FL is constrained by the limited number of available orthogonal wireless waveforms compared to the high dimensionality of modern neural networks. This necessitates gradient sparsification, typically implemented as a "Top-k" selection of the largest gradient magnitudes.

Standard Top-k selection is inherently greedy; it consistently ignores parameters with smaller magnitudes. Over time, these neglected parameters become stale, leading to reconstruction errors that stall convergence. While randomized alternatives exist, they often lack the directional precision required for efficient training. AgeTop-k addresses this by introducing a fairness mechanism based on update frequency.

The AgeTop-k Framework

The proposed AgeTop-k strategy utilizes a two-stage selection process managed by the edge server to optimize the use of limited wireless resources.

System architecture of the AgeTop-k OTA-FL framework, detailing the 7-step process from local training to age-aware gradient selection.

As illustrated in the architecture above, the process involves:

  1. Magnitude Filtering: The server identifies a candidate set of $r$ gradient entries with the largest magnitudes from its global tracking vector.
  2. Age Prioritization: From this candidate set, the server selects $k$ entries ($k < r$) with the highest Age of Information (AoI).

AoI tracks the time elapsed since a specific parameter was last updated. The server maintains an age vector that increments for skipped parameters and resets to zero upon update. By broadcasting this selection (the sparsification matrix $\hat{S}^t$) to clients, the server ensures that updates are both significant and overdue for a refresh.

Experimental Evaluation

The strategy was validated using two standard benchmarks: EMNIST (handwritten character recognition) and CIFAR-10 (object classification using ResNet-18). The experiments simulated a non-i.i.d. environment with 20 clients to reflect realistic edge computing challenges.

Test accuracy comparison across different sparsification strategies on EMNIST (top) and CIFAR-10 (bottom) over 500 communication rounds.

As shown in the performance plots, AgeTop-k consistently outperforms standard Top-k, Random-k, and rTop-k. On the CIFAR-10 dataset, AgeTop-k achieved a final test accuracy of approximately 69%, significantly higher than standard Top-k (~66%) and Age-k (~62%).

Key Insights

  • Mitigating Gradient Starvation: AgeTop-k acts as a fairness mechanism, ensuring every significant parameter eventually contributes to the global model, preventing the "starvation" of low-magnitude but essential coordinates.
  • The Primacy of Magnitude: Results indicate that selecting purely based on age (Age-k) performs as poorly as random selection. Magnitude must remain the primary filter, with age serving as the optimal tie-breaker among significant candidates.
  • Candidate Ratio Trade-off: There is a "Goldilocks zone" for the candidate set size ($r$). If $r$ is too small, the system reverts to greedy Top-k behavior. If $r$ is too large, the server may prioritize stale but irrelevant low-magnitude gradients, degrading accuracy.
  • Practicality: AgeTop-k requires minimal overhead, as the server only needs to maintain a simple vector of integers (ages), making it highly suitable for resource-constrained edge environments.

Conclusion

AgeTop-k provides a robust solution for the scalability challenges of OTA-FL. By balancing numerical significance with update freshness, it achieves superior convergence rates and accuracy compared to existing sparsification methods. Future work will explore adaptive tuning of the $r$ and $k$ hyperparameters to respond to real-time wireless channel conditions.

Summary updated

Abstract

Frequent parameter exchanges between clients and the edge server incur substantial communication overhead, posing a critical bottleneck in federated learning (FL). By exploiting the superposition property of wireless waveforms, over-the-air (OTA) computation enables simultaneous analog aggregation of local updates, thereby reducing communication latency and improving spectrum efficiency. However, its scalability is constrained by the limited number of available orthogonal waveform resources, which are typically far fewer than the model dimension. To address this, we propose AgeTop-$k$, an age-aware gradient sparsification strategy that performs compression through a two-stage selection process. Specifically, the edge server first selects candidate gradient entries based on their magnitudes, and then further prioritizes them according to the Age of Information (AoI), which quantifies the staleness of updates. AoI tracking is achieved efficiently by maintaining an age vector at the edge server. We derive theoretical convergence guarantees for non-convex loss functions and demonstrate the efficacy of AgeTop-$k$ through extensive simulations.
BibTeX
@misc{du2025age,
  author = {Ruihao Du and Zeshen Li and Howard H. Yang},
  title = {{Age-Aware Partial Gradient Update Strategy for Federated Learning Over the Air}},
  year = {2025},
  eprint = {2504.01357},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2504.01357},
}
Figure