Random Aggregate Beamforming for Over-the-Air Federated Learning in Large-Scale Networks

Chunmei Xu, Shengheng Liu, Yongming Huang, Björn E. Ottersten, Dusit Niyato

Introduction

Federated Learning (FL) has emerged as a transformative paradigm for training machine learning models across distributed edge devices, such as smartphones and IoT sensors, without requiring the sharing of raw data. Despite its privacy benefits, the communication overhead associated with transmitting high-dimensional model updates from thousands of devices to a central aggregator remains a primary bottleneck.

To mitigate this, Over-the-Air Computing (AirComp) has been proposed. AirComp leverages the physical superposition property of wireless channels to compute the weighted sum of local model updates directly as they are transmitted. However, the efficacy of AirComp relies heavily on precise aggregate beamforming and device selection to minimize the Mean Square Error (MSE) caused by channel fading and noise. In large-scale networks, finding the optimal beamforming vector and device subset is a complex combinatorial problem that typically requires full Channel State Information (CSI) for every device—a requirement that is practically impossible to meet in massive deployments.

Proposed Method: Random Aggregate Beamforming

This paper introduces a "random aggregate beamforming" scheme designed for large-scale FL networks. The core innovation lies in shifting away from computationally expensive optimization toward a statistically driven approach that eliminates the need for explicit CSI estimation.

1. Random Sampling and Selection

Instead of solving a complex optimization problem to determine the beamforming vector $\mathbf{m}$, the aggregator samples a vector uniformly from a complex unit sphere. This random vector is broadcast to all devices. Each device then estimates its own scalar "weighted channel gain" and reports it back to the aggregator. The aggregator subsequently selects the $S$ devices with the highest gains to participate in the current training round.

2. Asymptotic Optimality

Using extreme value theory and order statistics, the authors provide a theoretical proof that as the number of devices ($K$) in the network grows toward infinity, this random sampling approach approaches the optimal MSE. This suggests a "Law of Large Numbers" for beamforming: in a sufficiently dense network, a random beam is statistically guaranteed to align effectively with a large enough subset of devices.

3. Refined Random Beamforming

For scenarios with a more limited number of devices, the authors propose a refined method. The aggregator samples $N_m$ random vectors and selects the one that yields the best performance. This increases the computational complexity slightly to $O(N_m K)$, but significantly improves accuracy compared to a single random sample, while remaining far more efficient than traditional iterative optimization methods.

Experimental Evaluation

The proposed scheme was evaluated using standard image classification benchmarks in a non-i.i.d. (independent and identically distributed) setting, reflecting the data heterogeneity common in real-world FL.

  • MNIST: Handwritten digit recognition using a Multilayer Perceptron (MLP).
  • CIFAR-10: Color image classification using the ResNet18 architecture.

Key Results

  • Convergence: Simulations confirmed that the random method’s MSE approaches zero as the total number of devices $K$ increases for a fixed selection size $S$.
  • Performance in Mid-Range SNR: The refined random method frequently outperformed traditional benchmarks, such as the Difference of Convex (DC) algorithm, in terms of the number of devices successfully selected under specific error constraints.
  • Learning Accuracy: On the CIFAR-10 dataset, the proposed method achieved higher test accuracy faster than benchmarks. By incorporating more devices into each training round without exceeding the MSE threshold, the model benefited from more diverse data updates.
  • Computational Efficiency: The refined method (with 1,000 randomizations) demonstrated a 63.4% probability of being within a 0.011 margin of the absolute optimal objective in a 50-device scenario, while maintaining a low $O(K)$ complexity profile.

Conclusion

Random aggregate beamforming provides a low-complexity, CSI-free solution for large-scale AirComp-based Federated Learning. By leveraging the statistical properties of large networks, the method bypasses the prohibitive overhead of channel estimation and complex optimization. While the current model assumes homogeneous device capabilities and static channels, it establishes a robust foundation for future work, including the integration of learning factors like local dataset size and computational power into the selection policy.

Summary updated

BibTeX
@article{xu2024random,
  author = {Chunmei Xu and Shengheng Liu and Yongming Huang and Björn E. Ottersten and Dusit Niyato},
  title = {{Random Aggregate Beamforming for Over-the-Air Federated Learning in Large-Scale Networks}},
  journal = {IEEE Internet of Things Journal},
  year = {2024},
  url = {https://openreview.net/forum?id=uQq9srS627},
}
Figure