ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Models

Ahmed Salem, Yang Zhang, Mathias Humbert, Mario Fritz, Michael Backes

Abstract

Machine learning (ML) has become a core component of many real-world applications and training data is a key factor that drives current progress. This huge success has led Internet companies to deploy machine learning as a service (MLaaS). Recently, the first membership inference attack has shown that extraction of information on the training set is possible in such MLaaS settings, which has severe security and privacy implications. However, the early demonstrations of the feasibility of such attacks have many assumptions on the adversary, such as using multiple so-called shadow models, knowledge of the target model structure, and having a dataset from the same distribution as the target model's training data. We relax all these key assumptions, thereby showing that such attacks are very broadly applicable at low cost and thereby pose a more severe risk than previously thought. We present the most comprehensive study so far on this emerging and developing threat using eight diverse datasets which show the viability of the proposed attacks across domains. In addition, we propose the first effective defense mechanisms against such broader class of membership inference attacks that maintain a high level of utility of the ML model.

Introduction

Membership Inference Attacks (MIA)—the ability to determine if a specific data record was part of a model's training set—represent a critical privacy vulnerability. If an attacker can confirm a victim's data was used to train a specialized model (e.g., for a specific medical condition), they effectively uncover sensitive personal information.

Prior research suggested that MIA required significant resources, including multiple "shadow models" that mirrored the target's architecture and access to data from the same distribution. This paper, presented at NDSS 2019, challenges those assumptions, demonstrating that MIA is a far more universal and low-cost threat than previously realized.

Methodology and Adversary Models

The authors systematically relaxed the requirements for successful attacks by defining three increasingly restricted adversaries:

  1. Adversary 1 (Model Independent): Utilizes only a single shadow model and requires no knowledge of the target model's internal architecture or hyperparameters.
  2. Adversary 2 (Data Independent): Known as the "Data Transferring Attack," this adversary trains a shadow model on a dataset from a completely different domain (e.g., using text data to attack an image classifier).
  3. Adversary 3 (Unsupervised): The most restricted case, which uses no shadow models. It relies solely on the target model's output confidence (posterior probabilities). If the confidence for a given input exceeds a calibrated threshold, it is labeled a member.

To validate these attacks, the study utilized eight diverse datasets spanning images (MNIST, CIFAR), text (News), and tabular data (Purchase, Adult, Location).

The Role of Overfitting

A central insight of the research is that overfitting is the primary driver of MIA vulnerability. Models naturally behave differently on data they have seen during training versus unseen data, typically exhibiting higher confidence on the former.

Figure 3: Correlation between the level of overfitting in the target model and the precision/recall of the membership inference attack.

As shown in the scatter plots above, there is a direct positive correlation between the degree of overfitting and the success of the attack. This statistical signature of membership appears to be universal, explaining why attacks can transfer across different data distributions.

Experimental Results

The proposed relaxed attacks achieved performance levels comparable to high-assumption state-of-the-art methods. For instance, Adversary 1 reached 0.95 precision and recall on the CIFAR-100 dataset using only one shadow model.

Figure 1: Comparison of Precision and Recall between the proposed approach and the state-of-the-art (Shokri et al.) across eight diverse datasets.

Even more striking was the success of the Data Independent attack (Adversary 2). Using a shadow model trained on text data to attack an image classifier still yielded a precision of 0.94, proving that the "membership signature" is not domain-specific.

Proposed Defenses

The authors propose two primary defenses that mitigate MIA while preserving model utility:

  • Dropout: By randomly disabling neurons during training, dropout reduces the overfitting that attackers exploit. In tests on CIFAR-100, a 0.5 dropout ratio reduced attack precision from 0.95 to 0.61 with only a 1% loss in model accuracy.
  • Model Stacking: This ensemble technique trains multiple models on disjoint data subsets, preventing any single model from over-memorizing specific records.

Conclusion

This study proves that Membership Inference Attacks are a practical threat to almost any machine learning deployment, regardless of the attacker's knowledge of the model or data. While the threat is severe, common regularization techniques like dropout serve as effective, low-cost defenses that protect privacy without sacrificing the utility of the machine learning service.

Summary updated

Abstract

Machine learning (ML) has become a core component of many real-world applications and training data is a key factor that drives current progress. This huge success has led Internet companies to deploy machine learning as a service (MLaaS). Recently, the first membership inference attack has shown that extraction of information on the training set is possible in such MLaaS settings, which has severe security and privacy implications. However, the early demonstrations of the feasibility of such attacks have many assumptions on the adversary, such as using multiple so-called shadow models, knowledge of the target model structure, and having a dataset from the same distribution as the target model's training data. We relax all these key assumptions, thereby showing that such attacks are very broadly applicable at low cost and thereby pose a more severe risk than previously thought. We present the most comprehensive study so far on this emerging and developing threat using eight diverse datasets which show the viability of the proposed attacks across domains. In addition, we propose the first effective defense mechanisms against such broader class of membership inference attacks that maintain a high level of utility of the ML model.
BibTeX
@inproceedings{salem2019ml,
  author = {Ahmed Salem and Yang Zhang and Mathias Humbert and Mario Fritz and Michael Backes},
  title = {{ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Models}},
  booktitle = {NDSS},
  year = {2019},
  url = {https://openreview.net/forum?id=WsMs99LdPjo},
}
Figure