BEV-SGD: Best Effort Voting SGD Against Byzantine Attacks for Analog-Aggregation-Based Federated Learning Over the Air

Xin Fan, Yue Wang, Yan Huo, Zhi Tian

TL;DR

This paper introduces BEV-SGD, a novel defense mechanism for Federated Learning Over-the-Air (FLOA) against Byzantine attacks. By enabling workers to transmit at maximum power, BEV-SGD significantly improves convergence robustness compared to existing Channel Inversion (CI) methods, especially under strong adversarial conditions.

What Problem Does This Paper Address?

Federated Learning Over-the-Air (FLOA) offers a communication-efficient way to train models by allowing edge devices to transmit their updates simultaneously over a shared channel, where they are analogously aggregated. This "over-the-air computation" drastically reduces latency and communication overhead compared to traditional federated learning. However, this very mechanism of analog aggregation makes FLOA vulnerable to Byzantine attacks. In these attacks, malicious nodes can send manipulated updates to disrupt the learning process, potentially leading to divergence or a completely wrong model. Existing defenses for standard federated learning, which rely on inspecting individual model updates, are not applicable here because FLOA only provides a single aggregated signal. This paper addresses the critical need for robust defense mechanisms against Byzantine attacks specifically tailored for the unique challenges of FLOA.

Methods

The paper's core contribution is the proposal of a new power control policy called Best Effort Voting (BEV), integrated with Stochastic Gradient Descent (SGD). Unlike the prevalent Channel Inversion (CI) method, which adjusts transmit power based on channel conditions to equalize received signal amplitudes, BEV instructs normal workers to transmit their local gradients at their maximum transmit power, irrespective of their channel conditions. This "best effort" approach aims to overwhelm the influence of malicious updates.

The authors also rigorously analyze the system under the strongest possible Byzantine attack. This attack involves malicious nodes not only sending falsified gradients but also strategically choosing their transmit power to maximally disrupt convergence. They then derive theoretical convergence rates for both the proposed BEV-SGD and the existing CI method under these worst-case attack scenarios.

Datasets

The paper uses the MNIST dataset for evaluating the proposed method on an image classification task. They train a multi-layer perceptron (MLP) with 784 input neurons, 64 hidden neurons, and 10 output neurons. The dataset is split into 3000 training samples per worker.

Key Insights and Ideas

  • CI's Achilles' Heel: While Channel Inversion (CI) is effective in benign FLOA settings, its reliance on channel-specific power control makes it inherently weak against sophisticated Byzantine attacks. It struggles to maintain convergence when malicious actors actively try to mislead the aggregation.
  • BEV's Robustness: The "all-in" approach of BEV, where normal workers transmit at maximum power, acts as a powerful countermeasure. By amplifying the signal from honest nodes, it can drown out or mitigate the impact of malicious updates, even under strong attacks.
  • The Strongest Attack: Identifying and analyzing the "strongest attack" is crucial. This theoretical worst-case scenario provides a rigorous benchmark for evaluating the effectiveness of any proposed defense mechanism.
  • Trade-off in Benign Settings: BEV-SGD incurs a slight performance penalty (around 2%) in the absence of attacks compared to CI. This highlights a practical trade-off: robustness comes at a small cost in ideal conditions.

Key Results

  • Theoretical Guarantees: The paper provides theoretical convergence rate analyses for both CI and BEV under the strongest Byzantine attacks. These analyses demonstrate that BEV-SGD offers superior convergence behavior compared to CI under adversarial conditions.
  • Empirical Validation: Simulations on the MNIST dataset confirm the theoretical findings.
  • Without attacks, BEV-SGD shows a slight performance drop (around 2% in test accuracy) compared to CI and an ideal error-free scenario.
  • Under a single weak Byzantine attacker, both methods converge, but BEV-SGD converges faster with a larger learning rate.
  • Under a single strong Byzantine attacker (high channel gain), CI fails to converge, while BEV-SGD maintains convergence.
  • With multiple attackers, CI diverges when the number of attackers exceeds a certain threshold (related to the number of workers), while BEV-SGD continues to converge.

Limitations and Future Work

  • Non-IID Data: The current theoretical analysis assumes that local datasets are independent and identically distributed (i.i.d.). The paper acknowledges that handling non-i.i.d. data in the context of Byzantine attacks in FLOA is a more involved problem and leaves it for future work.
  • Model Complexity: The simulations use a relatively simple Multi-Layer Perceptron (MLP). Evaluating BEV-SGD on more complex deep neural networks would be a valuable extension.
  • Attack Model Specificity: While the paper analyzes the "strongest attack," real-world Byzantine attacks might exhibit different characteristics. Further research could explore robustness against a wider range of adversarial strategies.

Summary updated

Abstract

As a promising distributed learning technology, analog aggregation-based federated learning over the air (FLOA) provides high communication efficiency and privacy provisioning under the edge computing paradigm. When all edge devices (workers) simultaneously upload their local updates to the parameter server (PS) through commonly shared time-frequency resources, the PS obtains the averaged update only rather than the individual local ones. While such a concurrent transmission and aggregation scheme reduces the latency and communication costs, it unfortunately renders FLOA vulnerable to Byzantine attacks. Aiming at Byzantine-resilient FLOA, this article starts from analyzing the channel inversion (CI) mechanism that is widely used for power control in FLOA. Our theoretical analysis indicates that although CI can achieve good learning performance in the benign scenarios, it fails to work well with limited defensive capability against Byzantine attacks. Then, we propose a novel scheme called the best effort voting (BEV) power control policy that is integrated with stochastic gradient descent (SGD). Our BEV-SGD enhances the robustness of FLOA to Byzantine attacks, by allowing all the workers to send their local updates at their maximum transmit power. Under worst-case attacks, we derive the expected convergence rates of FLOA with CI and BEV power control policies, respectively. The rate comparison reveals that our BEV-SGD outperforms its counterpart with CI in terms of better convergence behavior, which is verified by experimental simulations.
BibTeX
@article{fan2022bev,
  author = {Xin Fan and Yue Wang and Yan Huo and Zhi Tian},
  title = {{BEV-SGD: Best Effort Voting SGD Against Byzantine Attacks for Analog-Aggregation-Based Federated Learning Over the Air}},
  journal = {IEEE Internet of Things Journal},
  year = {2022},
  doi = {10.1109/jiot.2022.3164339},
}
Figure