Topology-Independent Robustness of the Weighted Mean under Label Poisoning Attacks in Heterogeneous Decentralized Learning

Jie Peng, Weiyu Li, Stefan Vlaski, Qing Ling

Abstract

Robustness to malicious attacks is crucial for practical decentralized signal processing and machine learning systems. A typical example of such attacks is label poisoning, meaning that some agents possess corrupted local labels and share models trained on these poisoned data. To defend against malicious attacks, existing works often focus on designing robust aggregators; meanwhile, the weighted mean aggregator is typically considered a simple, vulnerable baseline. This paper analyzes the robustness of decentralized gradient descent under label poisoning attacks, considering both robust and weighted mean aggregators. Theoretical results reveal that the learning errors of robust aggregators depend on the network topology, whereas the performance of weighted mean aggregator is topology-independent. Remarkably, the weighted mean aggregator, although often considered vulnerable, can outperform robust aggregators under sufficient heterogeneity, particularly when: (i) the global contamination rate is smaller than the local contamination rate; (ii) the network of regular agents is disconnected; or (iii) the network of regular agents is sparse and the local contamination rate is high. Empirical results support our theoretical findings, highlighting the important role of network topology in the robustness to label poisoning attacks.

Introduction

Decentralized learning (DL) allows multiple devices (agents) to collaboratively train a model without a central server. While flexible, this setup is highly vulnerable to label poisoning attacks, where malicious agents use corrupted data labels to steer the global model astray. Conventional wisdom suggests that "robust aggregators"—such as Trimmed Mean or Median-based filters—are the gold standard for defense. However, this research reveals a surprising twist: the humble Weighted Mean (WeiMean), often dismissed as a vulnerable baseline, can actually outperform sophisticated robust aggregators when data is highly heterogeneous or the network topology is sparse.

Methodology

The study analyzes Decentralized Gradient Descent (DGD) under label poisoning, comparing WeiMean against a suite of state-of-the-art $(\rho, M)$-robust aggregators, including TriMean, FABA, CC, CG, and IOS. The core of the method is a rigorous theoretical convergence analysis deriving upper and lower bounds for learning errors, specifically looking at how these errors scale with global contamination (the total fraction of attackers) versus local contamination (the maximum fraction of attackers any single honest agent observes).

Validation was performed using standard computer vision benchmarks partitioned to simulate varying levels of data heterogeneity (i.i.d., mild non-i.i.d., and extreme non-i.i.d.):

  • MNIST (Softmax regression)
  • CIFAR-100 (ResNet18)
  • Tiny-ImageNet (ResNet34)

Theoretical Insights: The Topology Advantage

The most striking finding is the topology independence of the weighted mean. Its error depends only on the global average of poisoned data. In contrast, robust aggregators are topology-dependent; their ability to filter out bad actors is limited by how well the honest agents are connected to each other.

Robust aggregators work locally. If an honest agent is surrounded by many attackers (high local contamination), the robust aggregator fails. However, if the total number of attackers in the entire network is small (low global contamination), the weighted mean naturally dilutes the attack's impact across the whole system. Furthermore, in highly heterogeneous (non-i.i.d.) settings, robust aggregators often mistake "unique but honest" data for "malicious" data. The weighted mean avoids this trap by incorporating all perspectives, making it more stable when data varies wildly across the network.

Performance on CIFAR100 under varying levels of data heterogeneity.

Empirical Results

Theoretical bounds were confirmed through extensive experiments. The paper proves that the learning error for robust aggregators is $O(\delta_{max}^2 \xi^2)$, while for the weighted mean, it is $O(\delta^2 \xi^2)$ (where $\delta$ is global and $\delta_{max}$ is local contamination). When $\delta < \delta_{max}$, the mean is theoretically superior.

In practical tests on the "fan" graph—a sparse topology where local contamination is high for some nodes—the weighted mean consistently achieved higher classification accuracy than all five robust aggregators. While robust aggregators are often touted for helping agents reach a common model (consensus), the results showed that the weighted mean also achieves a stable consensus error, even under attack, provided the network remains connected.

Accuracy comparison on Two-castle and Fan graphs under label poisoning.

Conclusion

This research highlights that the "simple" weighted mean is a formidable baseline in decentralized learning, particularly when the network of honest agents is sparse or data heterogeneity is high. While robust aggregators remain valuable for defending against arbitrary Byzantine attacks, their performance is heavily tied to network topology. Future work will aim to bridge the theoretical gap between deterministic and stochastic optimization and explore the computational trade-offs of these aggregation methods in edge computing environments.

Summary updated

Abstract

Robustness to malicious attacks is crucial for practical decentralized signal processing and machine learning systems. A typical example of such attacks is label poisoning, meaning that some agents possess corrupted local labels and share models trained on these poisoned data. To defend against malicious attacks, existing works often focus on designing robust aggregators; meanwhile, the weighted mean aggregator is typically considered a simple, vulnerable baseline. This paper analyzes the robustness of decentralized gradient descent under label poisoning attacks, considering both robust and weighted mean aggregators. Theoretical results reveal that the learning errors of robust aggregators depend on the network topology, whereas the performance of weighted mean aggregator is topology-independent. Remarkably, the weighted mean aggregator, although often considered vulnerable, can outperform robust aggregators under sufficient heterogeneity, particularly when: (i) the global contamination rate (i.e., the fraction of poisoned agents for the entire network) is smaller than the local contamination rate (i.e., the maximal fraction of poisoned neighbors for the regular agents); (ii) the network of regular agents is disconnected; or (iii) the network of regular agents is sparse and the local contamination rate is high. Empirical results support our theoretical findings, highlighting the important role of network topology in the robustness to label poisoning attacks.
BibTeX
@misc{peng2026topology,
  author = {Jie Peng and Weiyu Li and Stefan Vlaski and Qing Ling},
  title = {{Topology-Independent Robustness of the Weighted Mean under Label Poisoning Attacks in Heterogeneous Decentralized Learning}},
  year = {2026},
  eprint = {2601.02682},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2601.02682},
}
Figure