Byzantine-Robust Federated Learning over Ring-All-Reduce Distributed Computing

Minghong Fang, Zhuqing Liu, Xuecen Zhao, Jia Liu

Abstract

Federated Learning (FL) typically relies on a centralized Server-Client (SC) architecture, which introduces communication bottlenecks and single points of failure. While Ring-All-Reduce (RAR) offers a bandwidth-optimal decentralized alternative, it is inherently vulnerable to Byzantine attacks due to the partial network view of each client. We present BRACE (Byzantine-robust ring-all-reduce), the first framework to provide robustness in RAR environments through 1-bit quantization and a novel majority-consensus mechanism.

Introduction: The Decentralization Dilemma

Traditional FL architectures struggle with scalability as the number of clients grows, often overwhelming the central server. RAR addresses this by organizing clients into a logical ring where they exchange gradient sub-vectors. However, this topology makes it difficult to detect "poisoned" updates from malicious actors. Because a client only sees what its immediate neighbor sends, it lacks the global perspective needed to filter out malicious data. BRACE overcomes this by adding layers of "skepticism" to the RAR process.

The BRACE Framework

BRACE operates in four distinct phases to ensure that only updates supported by a majority of clients are incorporated into the global model:

  1. Quantization: Instead of sending high-precision values, clients compute the element-wise sign (1-bit) of their local updates. This prevents attackers from using massive magnitudes to hijack the model.
  2. Share-Reduce: These quantized sub-vectors are passed around the ring and summed.
  3. Consensus: A thresholding mechanism ($\lambda$) is applied. A direction is only accepted if the sum of signs exceeds the threshold, representing a clear majority vote.
  4. Share-Only: The final consensus sign is distributed back to all clients so everyone updates their local models identically.
The four phases of the BRACE algorithm: Quantization, Share-Reduce, Consensus, and Share-Only.

Experimental Evaluation

The framework was evaluated on Fashion-MNIST and CIFAR-10 datasets with 100 clients, 20% of whom were malicious. BRACE demonstrated a convergence rate of $O(1/T)$, matching non-robust decentralized algorithms. In head-to-head comparisons, BRACE significantly outperformed existing defenses like Krum or Median under sophisticated attacks. For instance, under a "Krum attack," the Krum defense failed (0.90 error), while BRACE maintained a robust 0.19 error rate.

Performance and Robustness

A key strength of BRACE is its resilience to varying attack intensities and its bandwidth optimality. Because it compresses gradients to 1 bit before the final exchange, the communication cost remains $O(1)$ per round, ensuring scalability even as the network grows.

Test error rate of BRACE compared to other methods as the fraction of malicious clients increases.

As shown in the experimental results, BRACE maintains low error rates even as the fraction of malicious clients increases. However, like most Byzantine-robust systems, performance begins to degrade as the number of malicious actors approaches 50%. The system remains stable up to approximately 45% malicious participation, providing a significant security improvement over standard RAR.

Conclusion

BRACE provides a foundational approach for secure decentralized learning. By leveraging 1-bit quantization not just for compression but as a security shield, it achieves bandwidth optimality and robustness. Future work will explore complex network topologies beyond simple rings and alternative quantization schemes to further enhance decentralized FL security.

Summary updated

Abstract

Federated learning (FL) has gained attention as a distributed learning paradigm for its data privacy benefits and accelerated convergence through parallel computation. Traditional FL relies on a server-client (SC) architecture, where a central server coordinates multiple clients to train a global model, but this approach faces scalability challenges due to server communication bottlenecks. To overcome this, the ring-all-reduce (RAR) architecture has been introduced, eliminating the central server and achieving bandwidth optimality. However, the tightly coupled nature of RAR's ring topology exposes it to unique Byzantine attack risks not present in SC-based FL. Despite its potential, designing Byzantine-robust RAR-based FL algorithms remains an open problem. To address this gap, we propose BRACE (Byzantine-robust ring-all-reduce), the first RAR-based FL algorithm to achieve both Byzantine robustness and communication efficiency. We provide theoretical guarantees for the convergence of BRACE under Byzantine attacks, demonstrate its bandwidth efficiency, and validate its practical effectiveness through experiments. Our work offers a foundational understanding of Byzantine-robust RAR-based FL design.
BibTeX
@inproceedings{fang2025byzantine,
  author = {Minghong Fang and Zhuqing Liu and Xuecen Zhao and Jia Liu},
  title = {{Byzantine-Robust Federated Learning over Ring-All-Reduce Distributed Computing}},
  booktitle = {The Web Conference},
  year = {2025},
}
Figure