Reducing Risk of Model Inversion Using Privacy-Guided Training

Abigail Goldsteen, Gilad Ezov, Ariel FarkasharXiv 2020 · arXiv

Introduction

Machine learning models trained on sensitive datasets, such as medical records or financial histories, are susceptible to Model Inversion (MI) attacks. In these scenarios, an attacker leverages partial information and model outputs to infer sensitive missing attributes. Researchers from IBM have identified that high feature influence—the degree to which a model relies on a specific attribute—is a primary driver of MI vulnerability. To address this, they propose "privacy-guided training," a set of techniques designed to minimize the influence of sensitive features without significantly compromising model utility.

Privacy-Guided Training Methods

The proposed framework introduces three modifications to the training of tree-based models (Decision Trees, Random Forests, and AdaBoost):

  1. Weight-based Penalty: This method assigns sensitivity weights to features. During split selection (e.g., using Gini impurity), sensitive features incur a penalty, encouraging the model to prioritize non-sensitive attributes unless the sensitive one offers a substantial information gain.
  2. Limiting Split Levels: By restricting how high in a tree a sensitive feature can appear, the model ensures these features only impact smaller, more specific data subsets, making them harder for an attacker to generalize and extract.
  3. Limiting Split Counts: This approach caps the total number of times a sensitive feature is used. A Breadth-First Search (BFS) strategy is employed to distribute these splits evenly across the tree structure, preventing localized high-influence nodes.
Comparison of AdaBoost performance using weight-based penalties (Method 3.1) versus split level limits (Method 3.2).

Experimental Evaluation

The methods were evaluated using the Nursery dataset (12,958 samples) and the GSS Marital Happiness dataset (24,455 samples). The researchers simulated both black-box and white-box attacks to measure the effectiveness of their countermeasures.

Key Results and Discussion

A central finding of the study is that privacy protection does not necessarily require a significant "utility tax." In many instances, reducing the importance of sensitive features had a negligible impact on overall model accuracy.

Accuracy and importance loss metrics for Decision Trees and Random Forests. The table highlights stable accuracy (approx. 92%) even as the accumulated importance loss of sensitive features increases.

As shown in the results for the Nursery dataset, the researchers achieved a significant reduction in the importance of sensitive features while maintaining model accuracy around 92%. In the GSS dataset, reducing the "happiness" feature's importance from 0.315 to 0.01 caused white-box attack accuracy to drop from 80% to near-baseline levels (~60%). These results confirm that the success of inversion attacks is intrinsically linked to feature importance; by strategically driving down this importance, the risk of attribute leakage is mitigated.

Conclusion

Privacy-guided training offers a viable middle ground for deploying tree-based models on sensitive data. By focusing on reducing the influence of specific high-risk attributes rather than handicapping the entire model, researchers can protect individual privacy while maintaining high predictive performance. Future work will explore extending these influence-reduction strategies to neural network architectures and automating the identification of sensitive features.

Summary updated

Abstract

Machine learning models often pose a threat to the privacy of individuals whose data is part of the training set. Several recent attacks have been able to infer sensitive information from trained models, including model inversion or attribute inference attacks. These attacks are able to reveal the values of certain sensitive features of individuals who participated in training the model. It has also been shown that several factors can contribute to an increased risk of model inversion, including feature influence. We observe that not all features necessarily share the same level of privacy or sensitivity. In many cases, certain features used to train a model are considered especially sensitive and therefore propitious candidates for inversion. We present a solution for countering model inversion attacks in tree-based models, by reducing the influence of sensitive features in these models. This is an avenue that has not yet been thoroughly investigated, with only very nascent previous attempts at using this as a countermeasure against attribute inference. Our work shows that, in many cases, it is possible to train a model in different ways, resulting in different influence levels of the various features, without necessarily harming the model's accuracy. We are able to utilize this fact to train models in a manner that reduces the model's reliance on the most sensitive features, while increasing the importance of less sensitive features. Our evaluation confirms that training models in this manner reduces the risk of inference for those features, as demonstrated through several black-box and white-box attacks.
BibTeX
@misc{goldsteen2020reducing,
  author = {Abigail Goldsteen and Gilad Ezov and Ariel Farkash},
  title = {{Reducing Risk of Model Inversion Using Privacy-Guided Training}},
  year = {2020},
  eprint = {2006.15877},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2006.15877},
}
Figure