Benign Overparameterization in Membership Inference with Early Stopping

Jasper Tan, Daniel LeJeune, Blake Mason, Hamid Javadi, Richard G. Baraniuk

Abstract

Is overparameterization a privacy liability? This work investigates how the number of parameters in a classifier affects its vulnerability to membership inference (MI) attacks. While increasing parameters typically improves utility at the cost of privacy, we demonstrate that when paired with proper regularization—specifically early stopping—larger models can simultaneously achieve higher performance and better privacy. We provide theoretical evidence for this "blessing of dimensionality" using logistic regression with ridge regularization and empirically validate it across various neural network architectures and datasets.

Introduction

The prevailing trend in machine learning is toward increasingly massive, overparameterized models. While these models set new benchmarks for accuracy, they have long been viewed as a privacy risk. The conventional wisdom suggests that more parameters provide more "memory" for a model to store sensitive training data details, making it susceptible to Membership Inference (MI) attacks.

This research challenges that narrative. We demonstrate that overparameterization is not inherently a curse; with strategic regularization, it can be a "blessing" that improves both performance and privacy simultaneously.

The Privacy-Utility Tension

A Membership Inference (MI) attack attempts to determine if a specific data point was part of a model's training set. This is a critical concern for models trained on sensitive data, such as medical records. Previous studies suggested a fundamental trade-off: as parameters are added to improve generalization, the model becomes "leakier" because it treats training points differently than unseen test points.

We investigate whether this trade-off is unavoidable or if we can achieve high accuracy and high privacy simultaneously.

Methodology

Our approach combines rigorous mathematical theory with extensive empirical validation:

  1. Theoretical Analysis: We developed a novel leave-one-out analysis tool based on the Convex Gaussian Min-Max Theorem (CGMT) to characterize the output distribution of high-dimensional linear models.
  2. Regularization via Early Stopping: We focused on early stopping as a primary regularizer for deep neural networks. By monitoring the separation between training and non-training data distributions over time, we can identify the optimal point to halt training.
Evolution of log cross-entropy loss distributions for training and non-training data across different training epochs.

As shown in the figure above, as training progresses from 15 to 250 epochs, the loss distributions for training and non-training data become increasingly distinct. This separation is exactly what a membership inference attacker exploits. Early stopping prevents this extreme divergence while still allowing the model to learn generalizable features.

Key Insights: The Blessing of Dimensionality

  • Faster Learning vs. Slower Memorization: Highly overparameterized models have the capacity to learn underlying data structures faster than they memorize specific noise. If training is stopped appropriately, the model generalizes well without fitting to individual data points.
  • Superior Pareto Frontiers: When plotting Privacy vs. Accuracy, wider models often provide a strictly better trade-off (lower error for the same privacy level) than narrower ones.
  • Generalization Error vs. MI Vulnerability: Interestingly, MI vulnerability can continue to increase even after test error stabilizes. This occurs because the "loss gap" between training and test data continues to widen, providing more signal to an attacker even if accuracy remains constant.

Empirical Results

We evaluated our findings across diverse domains, including Purchase100 (2-layer NN), CIFAR-10 (ResNet-18), and Multi30K (Transformer).

Membership inference advantage and test error as a function of model width across different datasets.

Our results indicate that while increasing parameters alone can decrease privacy, doing so while tuning regularization (like early stopping) allows the model to reach performance-privacy states that smaller models cannot achieve. For instance, in the Multi30K dataset, we observe that as the width parameter increases, the test error continues to drop while the adversary's MI advantage eventually begins to decrease, demonstrating a simultaneous improvement in both metrics.

Conclusion

This work shows that the "curse" of overparameterization in privacy is not absolute. By leveraging the "blessing of dimensionality" through proper regularization, practitioners can build larger, more accurate models that are also more resistant to membership inference attacks. Future work will explore whether these benefits extend to white-box attacks and formal privacy frameworks like Differential Privacy.

This paper was presented at AISTATS 2023.

Summary updated

Abstract

Is overparameterization a privacy liability? In this work, we study the effect that the number of parameters has on a classifier's vulnerability to membership inference attacks. We first demonstrate how the number of parameters of a model can induce a privacy--utility trade-off: increasing the number of parameters generally improves generalization performance at the expense of lower privacy. However, remarkably, we then show that if coupled with proper regularization, increasing the number of parameters of a model can actually simultaneously increase both its privacy and performance, thereby eliminating the privacy--utility trade-off. Theoretically, we demonstrate this curious phenomenon for logistic regression with ridge regularization in a bi-level feature ensemble setting. Pursuant to our theoretical exploration, we develop a novel leave-one-out analysis tool to precisely characterize the vulnerability of a linear classifier to the optimal membership inference attack. We empirically exhibit this "blessing of dimensionality" for neural networks on a variety of tasks using early stopping as the regularizer.
BibTeX
@inproceedings{tan2023benign,
  author = {Jasper Tan and Daniel LeJeune and Blake Mason and Hamid Javadi and Richard G. Baraniuk},
  title = {{Benign Overparameterization in Membership Inference with Early Stopping}},
  booktitle = {International Conference on Artificial Intelligence and Statistics},
  year = {2023},
}
Figure