Alleviating Privacy Attacks via Causal Learning

Shruti Tople, Amit Sharma, Aditya Nori

Machine learning models, particularly deep neural networks (DNNs), are notorious for "memorizing" their training data. This tendency leads to a major security flaw known as Membership Inference Attacks (MIA). In an MIA, an adversary can determine if a specific individual's data was used to train a model simply by observing the model's predictions.

This paper reveals a powerful synergy between causal inference and data privacy. By training machine learning models to rely on causal relationships rather than mere statistical associations, the authors demonstrate that models not only generalize better to new distributions but also become significantly more resistant to membership inference and attribute inference attacks.

The Problem with Associational Learning

Most existing defenses against MIA, such as dropout or weight decay, assume that the training and testing data come from the same distribution. However, in the real world, distributions shift—for example, a medical model trained in one hospital might be deployed in another.

The authors point out that standard "associational" models often latch onto spurious correlations that change across environments. When the distribution shifts, the gap between how the model treats "seen" training data versus "unseen" test data widens. This generalization gap makes the model even more vulnerable to privacy leaks, as adversaries can easily distinguish between data the model memorized and data it hasn't seen.

Methods: Shifting to Causal Learning

The authors propose a fundamental shift from associational learning (finding any pattern that predicts the label) to causal learning (finding the features that actually cause the label).

  1. Causal Feature Selection: Instead of using all available features, a causal model ideally uses only the "parents" of the target variable in a structural causal graph. By isolating these direct causes, the model ignores spurious correlations.
Causal graphs illustrating feature selection, where causal models rely on the parents of the target variable.
  1. Theoretical Link to Privacy: The paper provides a mathematical proof showing that causal models have lower sensitivity. In the context of Differential Privacy (DP), sensitivity measures how much a single data point can change the model's parameters. Lower sensitivity means you can achieve a stronger privacy guarantee (a smaller $\epsilon$ value) with the same amount of added noise.
  2. Invariance: Because causal relationships are invariant—they don't change even if the background distribution of features changes—causal models exhibit much lower Out-of-Distribution (OOD) generalization error. This invariance acts as a natural privacy shield; if a model only learns stable causal links, it is less likely to memorize the idiosyncratic "noise" of specific training samples.

Experimental Setup

The researchers evaluated their approach on two types of data:

  • Bayesian Network Benchmarks: Four standard datasets (Child, Alarm, Sachs, and Water) where the true causal structure is known. This allowed for a "ground truth" comparison between causal and associational models.
  • Colored-MNIST: A variant of the classic digit dataset where colors are added as a "spurious" correlation (e.g., most 0s are red). This is a standard benchmark for testing if a model can ignore misleading associations and focus on the actual cause (the shape of the digit).

Key Results

The empirical results strongly support the theoretical claims, demonstrating that causal models drastically reduce the efficacy of privacy attacks.

  • Attack Accuracy: On the Bayesian network datasets, associational models (DNNs) showed up to 80% attack accuracy (where 50% is a random guess). In contrast, causal models stayed near 50%, effectively neutralizing the membership inference attack.
Comparison of model accuracy and membership inference attack accuracy between standard DNNs and Causal models.
  • Differential Privacy: The authors proved that for a fixed amount of noise, the privacy budget $\epsilon$ is always smaller (better) for causal models than for associational ones.
  • Colored-MNIST: Using Invariant Risk Minimization (IRM) to approximate causal learning, the attack accuracy was only 53%, compared to 66% for a standard Empirical Risk Minimization (ERM) model.
  • Attribute Inference: Causal models were shown to make it impossible for an adversary to infer "non-causal" sensitive features, as those features are never integrated into the model's decision-making process.

Limitations and Future Work

While promising, the approach has limitations. The most significant is the discovery of causal graphs. In many real-world, high-dimensional scenarios, we do not know the true causal structure. While the paper uses IRM as a proxy, discovering causal structures from complex data remains a difficult open problem.

Furthermore, there are inherent trade-offs. In some cases, causal models might have slightly lower predictive accuracy on the original training distribution compared to DNNs that exploit every available correlation. The authors view this as an acceptable trade-off for superior privacy and out-of-distribution generalization. Future work will investigate privacy guarantees in scenarios where causal information is insufficient or where there is significant selection bias in the observed data.

Summary updated

Abstract

Machine learning models, especially deep neural networks are known to be susceptible to privacy attacks such as membership inference where an adversary can detect whether a data point was used to t...
BibTeX
@inproceedings{tople2020alleviating,
  author = {Shruti Tople and Amit Sharma and Aditya Nori},
  title = {{Alleviating Privacy Attacks via Causal Learning}},
  booktitle = {International Conference on Machine Learning},
  year = {2020},
  url = {https://openreview.net/forum?id=IdOT_j-SmJ},
}
Figure