Comprehensive Privacy Analysis of Deep Learning: Passive and Active White-box Inference Attacks against Centralized and Federated Learning

Milad Nasr, Reza Shokri, Amir Houmansadr

Abstract

Deep neural networks (DNNs) are known to memorize specific details about their training data, creating significant privacy risks. This paper presents a comprehensive framework for white-box membership inference attacks (MIA), which aim to determine if a specific data record was part of a model's training set. By exploiting the internal parameters and gradient "footprints" left by the Stochastic Gradient Descent (SGD) algorithm, we demonstrate that deep learning models leak substantially more information than previously identified in black-box settings. Our analysis covers centralized and federated learning, passive and active attackers, and various levels of adversary knowledge, revealing that even well-generalized models remain highly vulnerable.

Introduction

Membership Inference Attacks (MIA) constitute a direct privacy violation; for instance, confirming a patient's record was used to train a disease-prediction model reveals their medical status. While prior research focused on black-box settings—where attackers only observe final predictions—this paper argues that such analysis underestimates the true risk. In modern contexts like Federated Learning (FL) or the use of open-source pre-trained models, attackers often have white-box access to internal parameters and gradients. We quantify these risks by designing novel inference algorithms tailored to the white-box setting.

A Taxonomy of Inference Attacks

To systematically evaluate privacy leakage, we categorize attacks based on four primary criteria: the level of observation, the target model type, the attack mode, and the attacker's prior knowledge.

Taxonomy of membership inference attacks across different criteria.
  • Observation: Distinguishes between black-box (output only) and white-box (access to weights $\mathbf{W}$ and intermediate activations).
  • Target: Covers stand-alone models (centralized) and federated learning environments.
  • Mode: Differentiates between passive observation and active manipulation of the training process.
  • Knowledge: Ranges from supervised (having a labeled subset of the target data) to unsupervised (clustering based on data distributions).

Methodology: Exploiting the Footprints of SGD

The core innovation of this work is the exploitation of the Stochastic Gradient Descent (SGD) algorithm. During training, SGD pushes the gradients of training samples toward zero to minimize loss. This leaves a distinguishable "footprint" on the model's gradients that is absent for unseen data.

To capture these footprints, we designed a modular neural network architecture for inference:

  • Feature Extraction: The attack processes the target record's gradients (per layer), activation values, model output, loss value, and one-hot encoded labels.
  • Component Networks: Gradients are analyzed via Convolutional Neural Networks (CNNs) to capture spatial correlations, while other features use Fully Connected Networks (FCNs).
  • Unsupervised Inference: In cases without labeled "member/non-member" examples, we employ an encoder-decoder approach to find data embeddings and cluster them to identify training set members.

Vulnerabilities in Federated Learning

In Federated Learning, multiple participants collaborate to train a global model by sharing parameter updates. This setting introduces unique vulnerabilities, particularly regarding the central aggregator and malicious participants.

The federated learning architecture and potential attack vectors.

We introduce an active attack for the FL setting. A malicious participant (or the aggregator) can perform gradient ascent on a target data point before sharing updates. This forces the global model to react more significantly to that specific data point in subsequent rounds, effectively "trapping" the record and making its membership status much easier to detect.

Key Results and Insights

Our evaluation across standard benchmarks (CIFAR100, Purchase100, Texas100) using architectures like ResNet and DenseNet yielded several critical insights:

  1. Generalization is not a Privacy Shield: High test accuracy does not prevent white-box leakage. Even when a model's predictions look similar for members and non-members, its internal gradients remain distinct. On a well-generalized DenseNet (CIFAR100), white-box attack accuracy reached 74.3%, compared to just 54.5% for black-box methods.
  2. Federated Learning Dominance: The central aggregator is in a powerful position, achieving 85.1% accuracy on CIFAR100. The active "isolating" attack further boosted this to 92.1%.
  3. Layer-wise Vulnerability: Later layers of a network leak more information than earlier layers, as they represent more complex, data-specific features.
  4. Unsupervised Success: Our unsupervised clustering attack outperformed previous state-of-the-art "shadow model" techniques, proving that attackers do not need a labeled training set to be effective.

Conclusion

This research demonstrates that deep learning models are significantly more susceptible to membership inference than previously thought, especially when internal model states are exposed. The findings highlight that standard generalization metrics are insufficient for privacy protection. Future work must focus on developing "privacy-preserving SGD" and formal mathematical bounds to mask these gradient footprints without sacrificing model utility.

Summary updated

Abstract

Deep neural networks are susceptible to various inference attacks as they remember information about their training data. We design white-box inference attacks to perform a comprehensive privacy analysis of deep learning models. We measure the privacy leakage through parameters of fully trained models as well as the parameter updates of models during training. We design inference algorithms for both centralized and federated learning, with respect to passive and active inference attackers, and assuming different adversary prior knowledge. We evaluate our novel white-box membership inference attacks against deep learning algorithms to trace their training data records. We show that a straightforward extension of the known black-box attacks to the white-box setting (through analyzing the outputs of activation functions) is ineffective. We therefore design new algorithms tailored to the white-box setting by exploiting the privacy vulnerabilities of the stochastic gradient descent algorithm, which is the algorithm used to train deep neural networks. We investigate the reasons why deep learning models may leak information about their training data. We then show that even well-generalized models are significantly susceptible to white-box membership inference attacks, by analyzing state-of-the-art pre-trained and publicly available models for the CIFAR dataset. We also show how adversarial participants, in the federated learning setting, can successfully run active membership inference attacks against other participants, even when the global model achieves high prediction accuracies.
BibTeX
@inproceedings{nasr2019comprehensive,
  author = {Milad Nasr and Reza Shokri and Amir Houmansadr},
  title = {{Comprehensive Privacy Analysis of Deep Learning: Passive and Active White-box Inference Attacks against Centralized and Federated Learning}},
  booktitle = {IEEE Symposium on Security and Privacy},
  year = {2019},
  url = {https://openreview.net/forum?id=meJ5A_sjXh},
}
Figure