Over-the-Air Federated Edge Learning with Error-Feedback One-Bit Quantization and Power Control

Yuding Liu, Dongzhu Liu, Guangxu Zhu, Qingjiang Shi, Caijun Zhong

TL;DR

This paper introduces EFOBDA, a communication-efficient framework for Federated Learning at the network edge. By combining aggressive one-bit gradient quantization with an "error-feedback" mechanism and optimized power control, the authors demonstrate that edge devices can train complex models over wireless channels with significantly higher accuracy and faster convergence than previous one-bit methods, nearly matching the performance of uncompressed systems.

What Problem Does This Paper Address?

Federated Edge Learning (FEEL) allows devices like smartphones or IoT sensors to train a shared machine learning model without uploading their raw, private data. However, two major bottlenecks exist:

  1. Communication Overhead: Modern models have millions of parameters. Transmitting these gradients individually from many devices creates a massive traffic jam.
  2. Wireless Distortion: "Over-the-Air" computation (AirComp) allows devices to transmit simultaneously, letting the wireless channel itself "sum" the gradients. While efficient, this is highly sensitive to channel fading and noise, which can scramble the aggregated model update.

Previous solutions used One-Bit Digital Aggregation (OBDA) to compress gradients to a single bit (sign-only). While this saves bandwidth, the extreme loss of information often leads to poor model accuracy and slow training. The authors set out to fix this "resolution vs. reliability" trade-off.

Fig. 1: The architecture of Air-FEEL.

Methods

The authors propose Error-Feedback One-Bit Broadband Digital Aggregation (EFOBDA). The core innovation lies in two areas:

  • Error Feedback (The "Memory" Trick): Instead of simply throwing away the information lost during one-bit quantization, each device stores this "quantization error" locally. In the next training round, this stored error is added back to the new gradient before the next round of quantization. This ensures that important gradient information isn't lost forever; it's just delayed.
  • Optimized Power Control: In a wireless environment, signals from different devices arrive with different strengths due to fading. The authors formulated a power control policy that minimizes "signal misalignment error." By solving a convex optimization problem, they derived a closed-form solution that tells each device exactly how much power to use to ensure their one-bit updates sum correctly at the server, even under strict battery/power constraints.

Datasets

The researchers evaluated their framework using two staple computer vision benchmarks:

  • MNIST: A dataset of 70,000 handwritten digits. They tested this in a non-i.i.d. (independent and identically distributed) setting, which is more realistic for federated learning where different users have different data distributions.
  • CIFAR-10: A more complex dataset of 60,000 color images across 10 classes (planes, cars, birds, etc.), using the ResNet18 architecture.

Key Insights and Ideas

  • The $\beta$ Trade-off: The authors introduced a feedback strength parameter ($\beta$). They discovered a crucial trade-off: increasing feedback strength reduces quantization error but can slow down initial convergence. Finding the "sweet spot" (around $\beta = 0.8$ in their tests) is key to performance.
  • Quantization doesn't have to be a dealbreaker: The theoretical analysis shows that with proper error feedback, a one-bit system can achieve a convergence rate comparable to a system using full-precision analog modulation.
  • Regularized Channel Inversion: The optimal power control policy resembles a "regularized channel inversion." If a device has a very weak channel, it doesn't just blast power (which would drain the battery); instead, the system balances signal alignment with power efficiency.
  • Scalability: The benefits of EFOBDA are most pronounced when the number of devices ($K$) is small or the signal-to-noise ratio (SNR) is low—scenarios where standard one-bit schemes typically fail.

Key Results

  • Accuracy Boost: On the MNIST dataset over fading channels, EFOBDA achieved nearly 95% test accuracy, whereas the standard one-bit OBDA scheme struggled to reach 70%.
  • Convergence Speed: EFOBDA reached high accuracy levels in roughly half the communication rounds required by standard one-bit methods.
  • Parity with Analog: Remarkably, the proposed one-bit scheme performed almost as well as "Broadband Analog Aggregation" (BAA), which uses much higher communication resources, proving that you can have both efficiency and precision.

Limitations and Future Work

While the results are promising, the authors note a few areas for further exploration:

  • Perfect CSI Assumption: The current power control model assumes the devices have "Perfect Channel State Information" (CSI). In real-world deployments, channel estimation is often imperfect and noisy.
  • Complex Fading Scenarios: The study focused on Rayleigh fading. Future work could investigate more complex environments, such as those with high mobility or significant interference.
  • Non-i.i.d. Complexity: While MNIST was tested in a non-i.i.d. fashion, the CIFAR-10 experiments were i.i.d. Exploring how EFOBDA handles highly skewed data on complex models like ResNet remains an open research direction.

Summary updated

Abstract

Over-the-air federated edge learning (Air-FEEL) is a communication-efficient framework for distributed machine learning using training data distributed at edge devices. This framework enables all edge devices to transmit model updates simultaneously over the entire available bandwidth, allowing for over-the-air aggregation. A one-bit digital over-the-air aggregation (OBDA) scheme has been recently proposed, featuring one-bit gradient quantization at edge devices and majority-voting based decoding at the edge server. However, the low-resolution one-bit gradient quantization slows down the model convergence and leads to performance degradation. On the other hand, the aggregation errors caused by fading channels in Air-FEEL is still remained to be solved. To address these issues, we propose the error-feedback one-bit broadband digital aggregation (EFOBDA) and an optimized power control policy. To this end, we first provide a theoretical analysis to evaluate the impact of error feedback on the convergence of FL with EFOBDA. The analytical results show that, by setting an appropriate feedback strength, EFOBDA is comparable to the Air-FEEL without quantization, thus enhancing the performance of OBDA. Then, we further introduce a power control policy by maximizing the convergence rate under instantaneous power constraints. The convergence analysis and optimized power control policy are verified by the experiments, which show that the proposed scheme achieves significantly faster convergence and higher test accuracy in image classification tasks compared with the one-bit quantization scheme without error feedback or optimized power control policy.
BibTeX
@misc{liu2023over,
  author = {Yuding Liu and Dongzhu Liu and Guangxu Zhu and Qingjiang Shi and Caijun Zhong},
  title = {{Over-the-Air Federated Edge Learning with Error-Feedback One-Bit Quantization and Power Control}},
  year = {2023},
  eprint = {2303.11319},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2303.11319},
}
Figure