A Blessing of Dimensionality in Membership Inference through Regularization

Jasper Tan, Daniel LeJeune, Blake Mason, Hamid Javadi, Richard G. Baraniuk

Abstract

Is overparameterization a privacy liability? In this work, we study the effect that the number of parameters has on a classifier's vulnerability to membership inference attacks. We first demonstrate how the number of parameters of a model can induce a privacy--utility trade-off: increasing the number of parameters generally improves generalization performance at the expense of lower privacy. However, remarkably, we then show that if coupled with proper regularization, increasing the number of parameters of a model can actually simultaneously increase both its privacy and performance, thereby eliminating the privacy--utility trade-off. Theoretically, we demonstrate this curious phenomenon for logistic regression with ridge regularization in a bi-level feature ensemble setting. Pursuant to our theoretical exploration, we develop a novel leave-one-out analysis tool to precisely characterize the vulnerability of a linear classifier to the optimal membership inference attack. We empirically exhibit this "blessing of dimensionality" for neural networks on a variety of tasks using early stopping as the regularizer.

Introduction

The machine learning community has embraced increasingly overparameterized models because they generalize remarkably well—a phenomenon often described as "double descent." However, privacy researchers have raised alarms: larger models have more capacity to memorize specific training examples. This leads to vulnerability against Membership Inference (MI) attacks, where an adversary tries to determine if a specific individual's data was used to train a model.

The central question addressed in this paper (presented at AISTATS 2023) is: Is there an unavoidable trade-off between a model's size and its privacy? While previous empirical studies suggested that larger models are easier to attack, this work investigates whether that vulnerability is a result of the model size itself or a failure to tune regularization appropriately as models grow.

Mechanism of Leakage and the Role of Regularization

Membership inference attacks typically exploit the difference in model behavior between training data and unseen test data. As training progresses, models tend to minimize loss on the training set more aggressively than on the test set, creating a gap that adversaries can exploit.

Evolution of training vs. non-training loss distributions over training epochs.

Regularization techniques, such as early stopping or ridge penalties, create a privacy-utility trade-off. Increasing regularization usually improves privacy by narrowing the gap between training and test distributions, but it eventually hurts accuracy. The authors demonstrate that overparameterized models learn the underlying structure of the data more quickly than they memorize individual points compared to smaller models.

The Blessing of Dimensionality

The core discovery is that for a larger model, the entire privacy-utility trade-off curve shifts favorably. A wide model and a narrow model might both be tuned to the same level of accuracy, but the wide model will require more regularization to get there, which in turn provides better protection against MI attacks.

Privacy (MI Advantage) and Utility (Test Error) trends across model widths for various datasets.

As shown in the empirical results across datasets like CIFAR10 and Multi30K, increasing the network width parameter $w$ consistently reduces test error. While the raw MI advantage may increase with width, the authors show that if you fix the privacy level, increasing network width reduces test error. Conversely, if you fix the test error, wider networks show lower MI vulnerability when properly regularized.

Theoretical Insights and Methodology

The authors tackle the problem through two lenses:

  1. Theoretical Analysis: They study high-dimensional logistic regression with ridge regularization using a novel leave-one-out analysis tool based on the Convex Gaussian Min-Max Theorem (CGMT). This allows them to precisely characterize the distribution of a model's output for a training point versus a test point in the asymptotic regime.
  2. Bi-level Feature Ensemble: They use a data model where "signal" exists in a low-dimensional subspace while "noise" is spread across a high-dimensional tail, mimicking the "benign overfitting" behavior of deep neural networks.

Conclusion

This paper challenges the conventional wisdom that larger models are inherently more "leaky." By leveraging the "blessing of dimensionality" through proper regularization, researchers can build models that are both more accurate and more private. While larger models come with higher computational costs, they offer a superior Pareto frontier for the privacy-utility trade-off.

Limitations

The study focuses on Membership Inference (MI) advantage rather than formal Differential Privacy (DP). While MI is a practical measure, it lacks the mathematical worst-case guarantees of DP. Additionally, the theoretical results rely on asymptotic assumptions, and the impact of other regularizers like dropout or batch normalization remains a subject for future work.

Summary updated

Abstract

Is overparameterization a privacy liability? In this work, we study the effect that the number of parameters has on a classifier's vulnerability to membership inference attacks. We first demonstrate how the number of parameters of a model can induce a privacy--utility trade-off: increasing the number of parameters generally improves generalization performance at the expense of lower privacy. However, remarkably, we then show that if coupled with proper regularization, increasing the number of parameters of a model can actually simultaneously increase both its privacy and performance, thereby eliminating the privacy--utility trade-off. Theoretically, we demonstrate this curious phenomenon for logistic regression with ridge regularization in a bi-level feature ensemble setting. Pursuant to our theoretical exploration, we develop a novel leave-one-out analysis tool to precisely characterize the vulnerability of a linear classifier to the optimal membership inference attack. We empirically exhibit this "blessing of dimensionality" for neural networks on a variety of tasks using early stopping as the regularizer.
BibTeX
@inproceedings{tan2023blessing,
  author = {Jasper Tan and Daniel LeJeune and Blake Mason and Hamid Javadi and Richard G. Baraniuk},
  title = {{A Blessing of Dimensionality in Membership Inference through Regularization}},
  booktitle = {International Conference on Artificial Intelligence and Statistics},
  year = {2023},
}
Figure