Over-the-Air Federated Learning via Weighted Aggregation

Seyed Mohammad Azimi-Abarghouyi, Leandros Tassiulas

Abstract

Federated Learning (FL) faces significant communication bottlenecks when scaling to numerous edge devices. Over-the-Air Computation (AirComp) addresses this by enabling simultaneous transmissions, yet typically requires perfect Channel State Information at the Transmitter (CSIT), which is difficult to obtain in practice. This paper introduces WAFeL (Weighted Over-the-Air Federated Learning), a scheme that shifts channel compensation to the server. By employing adaptive aggregation weights and a specialized receiver architecture, WAFeL handles fading and device heterogeneity without full CSIT. Evaluations on MNIST and CIFAR-10 show WAFeL outperforms existing blind schemes by up to 30%, nearing the performance of interference-free orthogonal communication.

Introduction

In Federated Learning (FL), many devices collaborate to train a model without sharing their raw data. However, communicating model updates from hundreds of devices to a central server is a massive bottleneck. Over-the-Air Computation (AirComp) is a promising solution that allows all devices to transmit simultaneously on the same frequency, leveraging the natural superposition property of wireless waves to sum updates mid-air.

System model of WAFeL showing over-the-air aggregation from heterogeneous devices.

Despite its potential, standard AirComp requires precise Channel State Information at the Transmitter (CSIT). Every device must estimate its channel and "pre-compensate" before sending, which creates high overhead and requires expensive hardware. Furthermore, most models assume computational homogeneity, ignoring that real-world devices have varying processing speeds. WAFeL addresses these challenges by shifting the burden of channel compensation from the devices to the server.

The WAFeL Framework

WAFeL introduces several key components to enable robust, blind wireless learning:

  • Blind Transmission: Devices transmit at constant power without knowing their specific channel gains. They only perform a "partial phase compensation" to ensure signals stay in the correct quadrant, which is significantly easier to implement than full CSIT.
  • Adaptive Aggregation Weights: Unlike standard FL, which uses fixed weights based on dataset size, WAFeL treats weights as optimizable parameters. The server adjusts these weights in every round to balance communication errors (noise/fading) against learning mismatch caused by heterogeneous batch sizes.
  • Heterogeneity-Aware Design: The algorithm explicitly accounts for different batch sizes across devices, ensuring that "stragglers" do not stall the global training process.
The specialized receiver architecture at the server for minimizing aggregation error.

As shown in the receiver architecture, the server uses a specialized equalization vector to process the real and imaginary parts of the received signal. This design is specifically optimized to minimize the Mean Squared Error (MSE) during the mid-air summation process.

Experimental Evaluation

The researchers evaluated WAFeL using MNIST and CIFAR-10 benchmarks under both i.i.d. and non-i.i.d. data distributions.

Key results include:

  • Accuracy Gains: In MNIST trials, WAFeL outperformed the leading CSIT-based scheme (BAA) by 15% and the leading blind scheme (GBMA) by 30%.
  • Near-Ideal Performance: WAFeL’s performance closely approached the "Orthogonal" case—the gold standard of zero interference—despite using significantly fewer communication resources.
  • Robustness: The system maintained high accuracy even when devices had a 3x difference in processing speeds.

Conclusion

WAFeL challenges the assumption that perfect channel knowledge at the device is necessary for high-performance AirComp. By optimizing at the receiver, the system achieves superior results with simpler transmitter hardware. Future work includes extending this framework to multi-antenna (MIMO) arrays and exploring power control mechanisms to further enhance performance.

Summary updated

Abstract

This paper introduces a new federated learning scheme that leverages over-the-air computation. A novel feature of this scheme is the proposal to employ adaptive weights during aggregation, a facet treated as predefined in other over-the-air schemes. This can mitigate the impact of wireless channel conditions on learning performance, without needing channel state information at transmitter side (CSIT). We provide a mathematical methodology to derive the convergence bound for the proposed scheme in the context of computational heterogeneity and general loss functions, supplemented with design insights. Accordingly, we propose aggregation cost metrics and efficient algorithms to find optimized weights for the aggregation. Finally, through numerical experiments, we validate the effectiveness of the proposed scheme. Even with the challenges posed by channel conditions and device heterogeneity, the proposed scheme surpasses other over-the-air strategies by an accuracy improvement of 15% over the scheme using CSIT and 30% compared to the one without CSIT.
BibTeX
@article{azimiabarghouyi2024over,
  author = {Seyed Mohammad Azimi-Abarghouyi and Leandros Tassiulas},
  title = {{Over-the-Air Federated Learning via Weighted Aggregation}},
  journal = {IEEE Trans. Wireless Communications},
  year = {2024},
  doi = {10.1109/twc.2024.3463754},
}
Figure