Decaf: Data Distribution Decompose Attack against Federated Learning

Zhiyang Dai, Chunyi Zhou, Anmin FuIEEE Trans. IFS 2024 · OpenReview

TL;DR

This paper introduces "Decaf," a novel privacy attack against Federated Learning (FL) that can precisely infer a user's local data distribution (i.e., the proportion of each class in their dataset). Decaf achieves this by analyzing local model gradient changes, offering a new threat beyond existing FL privacy attacks, and importantly, it operates passively and stealthily without impacting FL model performance.

What Problem Does This Paper Address?

Federated Learning (FL) is designed to train machine learning models collaboratively without centralizing user data, thereby enhancing privacy. However, existing privacy attacks on FL, such as Generative Adversarial Network (GAN) attacks, membership inference, property inference, and model inversion, primarily focus on inferring specific data samples or general properties. A significant gap remains in inferring the distribution of a user's local data, which can reveal sensitive information like local market item distribution or business competitiveness. This paper addresses this gap by proposing an attack that can accurately decompose this local data distribution.

Methods

Decaf is a passive privacy inference attack that leverages the observation that local model gradient changes during FL training are highly correlated with the underlying data distribution, specifically the proportion of each class. The attack proceeds in four main steps:

  1. Gradient Change Extraction: The server extracts gradient changes from the last fully connected layer of a victim user's local model. This layer is chosen because its neurons are strongly associated with specific classes.
  2. Null Classes Removal: The attack first identifies and removes "null classes" – classes that are not present in the victim user's local dataset. This is crucial for accurate decomposition and is achieved by observing that neurons corresponding to null classes will not exhibit positive gradient changes.
  3. Gradient Bases Construction: Using a small, auxiliary dataset (available to the server), the attack constructs "gradient bases." These bases simulate the expected gradient changes for each class under an idealized, evenly distributed scenario.
  4. Remaining Classes Decomposition: The attack formulates an optimization problem (specifically, linear regression) to infer the proportion of each non-null class. This is done by fitting the extracted gradient changes to the constructed gradient bases, using scalar factors to represent class proportions.

Datasets

The authors evaluated Decaf on five benchmark datasets:

  • MNIST: A dataset of 0-9 handwritten digits.
  • FASHION-MNIST: A dataset of clothing items, resembling commodity recommendation scenarios.
  • CIFAR-10: A dataset of 10 common object classes.
  • FER-2013: A face expression dataset, relevant for sensitive local user data.
  • SkinCancer: A dataset of dermoscopic images, containing private medical information.

Key Insights and Ideas

  • Gradient Magnitude as a Distribution Proxy: The core insight is that the magnitude of local model gradient changes directly reflects the proportion of data samples belonging to each class in a user's local dataset.
  • Null Classes are Detectable: Neurons in the last fully connected layer are class-specific. If a class is absent in a user's local data, its corresponding neurons will not show positive gradient changes, allowing for accurate identification of null classes.
  • Passive and Stealthy Attack: Decaf is designed to be entirely passive, meaning it doesn't alter the FL training process or the global model's accuracy. This makes it undetectable to the victim user.
  • Robustness to Non-IID Data: The attack is effective even when user data is non-independently and identically distributed (non-IID), a common scenario in FL.
  • Minimal Auxiliary Data Requirement: Decaf can function effectively with a very small auxiliary dataset (e.g., 2 samples per class) for constructing gradient bases.

Key Results

  • High Accuracy: Decaf achieves high accuracy in decomposing local user data distributions. When no null classes exist, the $L_{\infty}$ distance between the decomposed and ground truth distribution is consistently below 5%.
  • 100% Null Class Identification: Decaf achieves 100% accuracy in identifying a victim user's null classes, validated through formal proof.
  • Stealthiness: The attack does not impact the global model's accuracy or introduce significant latency, making it undetectable.
  • Generalization: Decaf performs well across diverse datasets and model architectures, including customized CNNs, VGG16, and ResNet18.
  • Robustness to Defenses: While differential privacy can degrade Decaf's performance, especially with non-IID data, it comes at the cost of significantly hurting the global model's utility.

Limitations and Future Work

  • Attack Stability: While Decaf works on single-round snapshots, its accuracy can decrease in later FL rounds as the global model converges. The authors recommend performing the attack in earlier stages of FL training for higher sensitivity.
  • Model Complexity: For very complex models (like VGG16 with a large number of neurons in the last layer), the $L_1$ distance can increase, though $L_{\infty}$ remains low.
  • Defenses: While differential privacy can mitigate Decaf, it significantly impacts FL utility. The paper suggests exploring other defenses like homomorphic encryption and secure multi-party computation, though these often have their own trade-offs.
  • Future Work: The paper implicitly suggests further research into more robust defenses against this type of data distribution inference attack and exploring its application in more complex FL scenarios.

Summary updated

Abstract

In contrast to prevalent Federated Learning (FL) privacy inference techniques such as generative adversarial networks attacks, membership inference attacks, property inference attacks, and model inversion attacks, we devise an innovative privacy threat: the Data Distribution Decompose Attack on FL, termed Decaf. This attack enables an honest-but-curious FL server to meticulously profile the proportion of each class owned by the victim FL user, divulging sensitive information like local market item distribution and business competitiveness. The crux of Decaf lies in the profound observation that the magnitude of local model gradient changes closely mirrors the underlying data distribution, including the proportion of each class. Decaf addresses two crucial challenges: accurately identify the missing/null class(es) given by any victim user as a premise and then quantify the precise relationship between gradient changes and each remaining non-null class. Notably, Decaf operates stealthily, rendering it entirely passive and undetectable to victim users regarding the infringement of their data distribution privacy. Experimental validation on five benchmark datasets (MNIST, FASHION-MNIST, CIFAR-10, FER-2013, and SkinCancer) employing diverse model architectures, including customized convolutional networks, standardized VGG16, and ResNet18, demonstrates Decaf's efficacy. Results indicate its ability to accurately decompose local user data distribution, regardless of whether it is IID or non-IID distributed. Specifically, the dissimilarity measured using $L_{\infty}$ distance between the distribution decomposed by Decaf and ground truth is consistently below 5\% when no null classes exist. Moreover, Decaf achieves 100\% accuracy in determining any victim user's null classes, validated through formal proof.
BibTeX
@article{dai2024decaf,
  author = {Zhiyang Dai and Chunyi Zhou and Anmin Fu},
  title = {{Decaf: Data Distribution Decompose Attack against Federated Learning}},
  journal = {IEEE Trans. Inf. Forensics Secur.},
  year = {2024},
  url = {https://openreview.net/forum?id=hBPgwytwAy},
}
Figure