Alleviating Privacy Attacks via Causal Learning

Shruti Tople, Amit Sharma, Aditya Nori

Abstract

Machine learning models, particularly deep neural networks (DNNs), are increasingly vulnerable to privacy attacks such as Membership Inference Attacks (MIA), where adversaries detect if a specific data point was used for training. These risks are heightened under distribution shifts, where models rely on spurious correlations that do not hold in new environments. This paper proposes causal learning as a robust defense. By training models on stable cause-and-effect relationships rather than simple associations, we demonstrate that causal models inherently provide stronger differential privacy guarantees and reduce MIA success to near-random levels. Experiments on Bayesian networks and the Colored-MNIST dataset confirm that while associational models suffer up to 80% attack accuracy, causal models remain resilient.

The Vulnerability of Associational Models

Deep learning models are notoriously "leaky" because they tend to overfit by latching onto patterns specific to their training set that do not exist in the broader population. Standard regularization techniques often fail when the training and testing data come from different distributions (e.g., a medical model trained in one hospital but deployed in another). Under these distribution shifts, traditional associational models become even more vulnerable to privacy leaks as they rely on "spurious" correlations—features that are statistically associated with the target but not causally responsible for it.

Methodology: From Associations to Causal Mechanisms

The authors propose shifting from associational models, which use all available features to minimize error, to causal models that only utilize features with a direct causal relationship to the outcome.

  1. Causal Invariance: Causal relationships are, by definition, invariant across different environments. A causal model focuses on the underlying mechanism generating the output, making it robust to changes in the input distribution.
  2. Sensitivity and Differential Privacy: The paper mathematically proves that causal learning functions have lower $l_1$-sensitivity than associational ones. In Differential Privacy (DP), sensitivity measures how much a single data point can alter model parameters. Lower sensitivity allows for a smaller privacy budget ($\epsilon$), providing stronger guarantees for the same amount of noise.
  3. Markov Blanket vs. Causal Parents: While standard models might use the entire Markov Blanket of a variable (including children and spouses), causal models restrict themselves to the "parents" of the target variable to ensure stability.
Causal graph structures illustrating the relationship between target variable Y and various feature types, including parents (X_PA) and the Markov Blanket.

Key Insights: Privacy through Generalization

The core thesis is that privacy and out-of-distribution (OOD) generalization are two sides of the same coin. A model that truly understands the underlying causal mechanism does not need to "memorize" specific training samples to perform well. Associational models are "greedy" and will incorporate any feature, including noise, to lower training loss. These coincidental patterns are exactly what MIA exploits to identify training set members.

Experimental Results

The researchers evaluated their approach using simulated Bayesian networks (Child, Alarm, Sachs, and Water) and the Colored-MNIST dataset, where color is a spurious correlation and digit shape is the causal feature.

  • MIA Accuracy: On Bayesian networks, associational DNNs showed up to 80% attack accuracy under distribution shifts. Causal models stayed near 50% (random guess), effectively neutralizing the attack.
  • Generalization Gap: As shown in the performance metrics, DNNs exhibit a significant gap between training accuracy and test accuracy on shifted distributions ($P^*$). Causal models, however, maintain consistent performance across training and various test distributions, leaving no "prediction gap" for attackers to exploit.
Comparison of accuracy between DNN and Causal models across different dataset sizes and distributions (P vs. P*).

Conclusion and Future Work

Causal models achieve high privacy not by adding massive amounts of accuracy-degrading noise, but by being "smarter" about feature selection. However, the primary limitation remains causal discovery: identifying the true causal graph for complex, high-dimensional real-world datasets is challenging. Future work will explore privacy guarantees when the causal structure is only partially known or when significant selection bias exists in the training data.

Summary updated

Abstract

Machine learning models, especially deep neural networks have been shown to be susceptible to privacy attacks such as membership inference where an adversary can detect whether a data point was used for training a black-box model. Such privacy risks are exacerbated when a model's predictions are used on an unseen data distribution. To alleviate privacy attacks, we demonstrate the benefit of predictive models that are based on the causal relationships between input features and the outcome. We first show that models learnt using causal structure generalize better to unseen data, especially on data from different distributions than the train distribution. Based on this generalization property, we establish a theoretical link between causality and privacy: compared to associational models, causal models provide stronger differential privacy guarantees and are more robust to membership inference attacks. Experiments on simulated Bayesian networks and the colored-MNIST dataset show that associational models exhibit upto 80% attack accuracy under different test distributions and sample sizes whereas causal models exhibit attack accuracy close to a random guess.
BibTeX
@inproceedings{tople2020alleviating,
  author = {Shruti Tople and Amit Sharma and Aditya Nori},
  title = {{Alleviating Privacy Attacks via Causal Learning}},
  booktitle = {International Conference on Machine Learning},
  year = {2020},
  url = {https://openreview.net/forum?id=IdOT_j-SmJ},
}
Figure