TL;DR
Decentralized Learning (DL) is often touted as a more private alternative to Federated Learning (FL) because it eliminates the central server. This paper systematically debunks that claim, demonstrating that DL actually increases the attack surface and enables both passive and active adversaries to perform devastating privacy attacks that are impossible for standard users in a federated setup.
What Problem Does This Paper Address?
Collaborative machine learning allows multiple parties to train a shared model without exchanging their raw, private datasets. The two dominant paradigms are Federated Learning (FL), which relies on a central parameter server to aggregate updates, and Decentralized Learning (DL), where users communicate in a peer-to-peer (P2P) fashion.
Proponents of DL argue that removing the central server solves communication bottlenecks and enhances privacy by giving users control over who they talk to. However, the authors of this paper identify a critical gap: while the efficiency of DL has been well-studied, its privacy guarantees have largely been taken on faith. They set out to answer whether decentralization actually helps or hurts user privacy.
Methods
The authors evaluate the privacy of the D-PSGD protocol (a representative DL framework) against both passive and active adversaries. They introduce several novel attack techniques:
- Functional Marginalization: A passive attack where an adversary uses "system knowledge"—the ability to see individual updates from multiple neighbors—to isolate a specific victim's contribution by subtracting an approximation of the other neighbors' influence.
- Echo Attack: An active attack where the adversary "echoes" a victim's own update back to them. This forces the victim's model to overfit severely on its local data, making it much easier to perform membership inference.
- State-Override Attack: A powerful active attack where an adversary uses knowledge of a victim's neighborhood to craft a malicious update that cancels out all other honest contributions. This allows the attacker to essentially "remote control" the victim's local model parameters.
They compare these DL-specific threats against standard FL baselines (FedAVG) to quantify the "privacy gap" created by decentralization.
Datasets
The study evaluates these attacks across diverse domains using standard benchmarks:
- Computer Vision: CIFAR-10, CIFAR-100, and STL10.
- Natural Language Processing (NLP): The IMDB-reviews dataset for text classification.
Key Insights and Ideas
- Local Generalization vs. Global Generalization: In FL, every update is averaged into a global model, providing a degree of "privacy by crowd." In DL, models are sparse; a user's local model is dominated by their own data and their immediate neighbors. This "local generalization" makes their updates far more revealing.
- The Curse of System Knowledge: In FL, users only see the final aggregate. In DL, users see raw updates from every neighbor. This extra information allows attackers to "unmix" the contributions of different users.
- The Inherent Conflict: The paper reveals a fundamental trade-off. To reduce the risks of local generalization, you need a denser network (more neighbors). But more neighbors increase the "system knowledge" available to an attacker.
- Efficiency vs. Privacy: To reach the same privacy level as FL, a DL network must be fully connected and use expensive Secure Aggregation. At that point, all the efficiency gains of decentralization are lost.
Key Results
- Passive Attacks: Membership Inference Attacks (MIA) are significantly more accurate in DL than in FL. Before the system reaches consensus, a passive DL neighbor can infer training set membership with much higher confidence than a federated user.
- Active Control: The State-Override attack is 100% effective. If an attacker is connected to all of a victim's neighbors, they can completely dictate the victim's local model state in a single round, matching the power of a malicious central server in FL.
- Echo Attack Impact: The Echo attack can increase the generalization error of a victim's model by up to 4x, leading to a massive spike in privacy leakage that persists even after the rest of the network reaches consensus.
Limitations and Future Work
The authors acknowledge that their attacks are most effective when the adversary has some knowledge of the network topology or can choose their neighbors. While they demonstrate a "neighbor-discovery trick" to learn local topology, highly dynamic or restricted networks might mitigate some of these risks.
The paper concludes that current DL protocols are fundamentally less private than FL. Future work must focus on developing new design principles—perhaps involving topology-aware secure aggregation or decentralized differential privacy—that can bridge this privacy gap without sacrificing the performance benefits that make decentralized learning attractive in the first place.
Summary updated