On the Privacy of Decentralized Machine Learning

Dario Pasquini, Mathilde Raynal, Carmela Troncoso

TL;DR

Decentralized Learning (DL) is often touted as a more private alternative to Federated Learning (FL) because it eliminates the central server. This paper systematically debunks that claim, demonstrating that DL actually increases the attack surface and enables both passive and active adversaries to perform devastating privacy attacks that are impossible for standard users in a federated setup.

What Problem Does This Paper Address?

Collaborative machine learning allows multiple parties to train a shared model without exchanging their raw, private datasets. The two dominant paradigms are Federated Learning (FL), which relies on a central parameter server to aggregate updates, and Decentralized Learning (DL), where users communicate in a peer-to-peer (P2P) fashion.

Proponents of DL argue that removing the central server solves communication bottlenecks and enhances privacy by giving users control over who they talk to. However, the authors of this paper identify a critical gap: while the efficiency of DL has been well-studied, its privacy guarantees have largely been taken on faith. They set out to answer whether decentralization actually helps or hurts user privacy.

Figure 1: Schematic representation of the decentralized learning and federated learning protocols. “PS” stands for Parameter Server.

Methods

The authors evaluate the privacy of the D-PSGD protocol (a representative DL framework) against both passive and active adversaries. They introduce several novel attack techniques:

  • Functional Marginalization: A passive attack where an adversary uses "system knowledge"—the ability to see individual updates from multiple neighbors—to isolate a specific victim's contribution by subtracting an approximation of the other neighbors' influence.
  • Echo Attack: An active attack where the adversary "echoes" a victim's own update back to them. This forces the victim's model to overfit severely on its local data, making it much easier to perform membership inference.
  • State-Override Attack: A powerful active attack where an adversary uses knowledge of a victim's neighborhood to craft a malicious update that cancels out all other honest contributions. This allows the attacker to essentially "remote control" the victim's local model parameters.

They compare these DL-specific threats against standard FL baselines (FedAVG) to quantify the "privacy gap" created by decentralization.

Datasets

The study evaluates these attacks across diverse domains using standard benchmarks:

  • Computer Vision: CIFAR-10, CIFAR-100, and STL10.
  • Natural Language Processing (NLP): The IMDB-reviews dataset for text classification.

Key Insights and Ideas

  • Local Generalization vs. Global Generalization: In FL, every update is averaged into a global model, providing a degree of "privacy by crowd." In DL, models are sparse; a user's local model is dominated by their own data and their immediate neighbors. This "local generalization" makes their updates far more revealing.
  • The Curse of System Knowledge: In FL, users only see the final aggregate. In DL, users see raw updates from every neighbor. This extra information allows attackers to "unmix" the contributions of different users.
  • The Inherent Conflict: The paper reveals a fundamental trade-off. To reduce the risks of local generalization, you need a denser network (more neighbors). But more neighbors increase the "system knowledge" available to an attacker.
  • Efficiency vs. Privacy: To reach the same privacy level as FL, a DL network must be fully connected and use expensive Secure Aggregation. At that point, all the efficiency gains of decentralization are lost.
Figure 2: Average loss of u1’s local model computed on every local training set.

Key Results

  • Passive Attacks: Membership Inference Attacks (MIA) are significantly more accurate in DL than in FL. Before the system reaches consensus, a passive DL neighbor can infer training set membership with much higher confidence than a federated user.
  • Active Control: The State-Override attack is 100% effective. If an attacker is connected to all of a victim's neighbors, they can completely dictate the victim's local model state in a single round, matching the power of a malicious central server in FL.
  • Echo Attack Impact: The Echo attack can increase the generalization error of a victim's model by up to 4x, leading to a massive spike in privacy leakage that persists even after the rest of the network reaches consensus.

Limitations and Future Work

The authors acknowledge that their attacks are most effective when the adversary has some knowledge of the network topology or can choose their neighbors. While they demonstrate a "neighbor-discovery trick" to learn local topology, highly dynamic or restricted networks might mitigate some of these risks.

The paper concludes that current DL protocols are fundamentally less private than FL. Future work must focus on developing new design principles—perhaps involving topology-aware secure aggregation or decentralized differential privacy—that can bridge this privacy gap without sacrificing the performance benefits that make decentralized learning attractive in the first place.

Summary updated

Abstract

In this work, we carry out the first, in-depth, privacy analysis of Decentralized Learning -- a collaborative machine learning framework aimed at addressing the main limitations of federated learning. We introduce a suite of novel attacks for both passive and active decentralized adversaries. We demonstrate that, contrary to what is claimed by decentralized learning proposers, decentralized learning does not offer any security advantage over federated learning. Rather, it increases the attack surface enabling any user in the system to perform privacy attacks such as gradient inversion, and even gain full control over honest users' local model. We also show that, given the state of the art in protections, privacy-preserving configurations of decentralized learning require fully connected networks, losing any practical advantage over the federated setup and therefore completely defeating the objective of the decentralized approach.
BibTeX
@inproceedings{pasquini2023privacy,
  author = {Dario Pasquini and Mathilde Raynal and Carmela Troncoso},
  title = {{On the Privacy of Decentralized Machine Learning}},
  booktitle = {SP},
  year = {2023},
  doi = {10.1109/sp46215.2023.10179291},
}
Figure