Abstract
As decentralized machine learning gains traction, Gossip Learning (GL) has emerged as a robust alternative to Federated Learning (FL) by eliminating the need for a central server. However, this peer-to-peer architecture introduces unique security challenges, particularly regarding Byzantine participants. This paper evaluates the resilience of GL against data poisoning attacks, specifically focusing on the Partitioned Token Gossip Learning Algorithm (PTGLA). By simulating backdoor attacks across various network topologies and churn conditions, we identify that while GL exhibits inherent resilience through model dilution, it remains highly vulnerable to targeted attacks on network hubs and specific topological configurations.
Introduction
Traditional Machine Learning (ML) relies on centralized data collection, which poses significant privacy risks. Federated Learning (FL) mitigated some of these risks by allowing local training on user devices, yet it remains dependent on a central parameter server—a single point of failure and a potential bottleneck.
Gossip Learning (GL) decentralizes the process further by removing the central authority entirely. In a GL framework, nodes exchange model parameters directly with their immediate neighbors. While this enhances system robustness and privacy, it exposes the network to malicious actors, or "Byzantine nodes," who seek to compromise the collective model through data poisoning. While poisoning has been extensively studied in server-based FL, its impact on the chaotic, dynamic environment of peer-to-peer GL—characterized by constant node "churn"—remains under-explored.
Methodology
This study focuses on the Partitioned Token Gossip Learning Algorithm (PTGLA), a state-of-the-art GL protocol designed for efficiency. PTGLA utilizes two primary mechanisms:
- Model Partitioning ($S$): The model is divided into $S$ segments, allowing nodes to exchange smaller updates rather than the full parameter set, thereby optimizing bandwidth.
- Token-Based Regulation: A token system manages message flow to prevent network congestion.
To evaluate security, the researchers extended the gossipy simulator with a "poison injector" module. The primary attack vector tested was a Backdoor Attack using the MNIST dataset. In this scenario, malicious nodes modify a subset of their local data by adding a 9-pixel trigger pattern to images of the digit "4" and relabeling them as "0." The goal is to "brainwash" the network into misclassifying triggered images while maintaining high accuracy on clean data.
The study compared two attacker placement strategies:
- Random Strategy: Malicious nodes are distributed uniformly across the network.
- Classical Strategy: Attackers specifically target "hubs"—nodes with the highest degree of connectivity in the network graph.
Key Insights and Analysis
The Partitioning Dilution Effect
One of the most significant findings is the role of the partition parameter ($S$). In PTGLA, increasing the number of partitions acts as a security mechanism. When a model is split into many small pieces, malicious updates from a single node are "diluted" by the influx of clean updates from honest neighbors. The simulations demonstrated that high $S$ values kept backdoor success rates between 0.1 and 0.2, whereas lower $S$ values allowed the poison to take hold more effectively.
Topology and Hub Vulnerability
Network structure is a primary determinant of GL resilience. In networks following a Zipf distribution—where a small number of hubs possess the majority of connections—the system is highly susceptible to the "Classical" attack strategy. By compromising these high-degree hubs, attackers can propagate poisoned updates with devastating efficiency. In these scenarios, the researchers observed a 38% drop in standard classification accuracy and a 48% increase in backdoor success compared to random attacker placement.
The Dual Role of Churn
Node churn—the frequency with which participants join or leave the network—presents a paradoxical effect on GL. While churn typically hinders learning speed, it also serves as a temporary defensive barrier. In environments where nodes were offline 80% of the time, the spread of poisoned updates was significantly retarded. Although honest nodes eventually converged to a functional model, the churn-heavy environment provided a buffer that prevented the rapid "epidemic" spread of the backdoor trigger.
Results
Experimental results in a 150-node network with 45 attackers highlighted the following:
- Attack Efficacy: The Classical attack on Zipf-based topologies was the most successful, nearly achieving a perfect backdoor (forcing honest nodes to misclassify almost all triggered images) while simultaneously degrading the performance of the legitimate task.
- Convergence: Despite the presence of attackers, honest nodes in most configurations eventually reached a state of convergence, though the learning curve was significantly flatter in the presence of churn and poisoning compared to clean environments.
Conclusion and Future Work
This research provides a foundational methodology for assessing the security of decentralized learning. The results suggest that while Gossip Learning possesses inherent defensive properties—such as partition-based dilution and churn-induced delay—it is not immune to sophisticated topological attacks.
Future research should expand beyond the Logistic Regression models and MNIST dataset used here to evaluate more complex deep learning architectures. Furthermore, there is a critical need for the development of active defense mechanisms, such as outlier detection or robust aggregation rules, that allow individual nodes to identify and filter poisoned gossip from their peers.
Summary updated