Rethinking Improved Privacy-Utility Trade-off with Pre-existing Knowledge for DP Training

Yu Zheng, Wenchao Zhang, Yonggang Zhang, Wei Song, Kai Zhou, Bo Han

Abstract

Differential privacy (DP) provides a provable framework for protecting individuals by customizing a random mechanism over a privacy-sensitive dataset. However, standard differentially private stochastic gradient descent (DP-SGD) often suffers from significant utility loss. This loss stems from the injection of homogeneous noise, which contaminates all gradient elements regardless of their importance to model updates. In this work, we propose DP-Hero (Differential Privacy with Heterogeneous Noise), a framework that leverages knowledge encoded in previously trained model parameters to guide the allocation of heterogeneous noise. By utilizing the statistical properties of the model's current state—which is already DP-protected—DP-Hero achieves enhanced utility and faster convergence. Our experiments on MNIST and CIFAR-10 demonstrate that DP-Hero significantly outperforms state-of-the-art methods, improving training accuracy by 18% to 47% under the same privacy constraints.

Introduction

As deep learning models become increasingly integrated into sensitive domains, the risk of membership-level privacy leakage has become a primary concern. Differentially private stochastic gradient descent (DP-SGD) has emerged as the standard defense, safeguarding training data by adding random Gaussian noise to gradient updates during backpropagation.

Despite its theoretical strengths, DP-SGD typically results in a steep decline in model accuracy. The root cause of this utility degradation is the "blind" nature of the noise: standard DP-SGD applies independent and identically distributed (i.i.d.) homogeneous noise to every gradient component. This approach treats every parameter in a neural network as equally significant, often drowning out vital signals in high-importance features while over-protecting less critical ones. We argue that the path to a better privacy-utility trade-off lies in breaking this homogeneity.

The DP-Hero Framework

DP-Hero introduces a stateful, heterogeneous noise allocation strategy. Rather than applying uniform noise, the framework scales noise based on the specific learning requirements of the model at each iteration.

Leveraging Pre-existing Knowledge

A key innovation of DP-Hero is its use of "pre-existing knowledge." At each training iteration $t$, the framework analyzes the model parameters from the previous step ($W^{(t-1)}$). Because these parameters were generated using gradients already protected by DP, they can be reused to guide future noise allocation without consuming any additional privacy budget. This allows the framework to extract statistical insights "for free."

PCA-Guided Noise Allocation

To determine where noise should be allocated, DP-Hero performs Principal Component Analysis (PCA) on the parameter state. This process identifies the "subspace" where the model is currently concentrating its learning. The framework then generates a guidance matrix ($SVec$) based on these PCA results.

This matrix acts as a sophisticated filter that scales the noise distribution. Specifically, it directs the system to inject smaller amounts of noise into high-variance directions—where the learning signal is strongest and most robust—and larger amounts of noise into low-variance directions. This aligns the noise distribution with the actual anisotropic shape of the data features, ensuring that the most critical gradient components remain as clear as possible.

Experimental Evaluation

We evaluated DP-Hero against standard DP-SGD and several state-of-the-art baselines using two benchmark computer vision datasets: MNIST and CIFAR-10.

  • MNIST: DP-Hero achieved over 98% accuracy, a notable improvement over the 95-97% typically seen with standard DP-SGD.
  • CIFAR-10: Training on CIFAR-10 from scratch under DP constraints is notoriously difficult. While many baselines struggle to reach 50% accuracy, DP-Hero reached over 65% accuracy.

Beyond raw accuracy, DP-Hero demonstrated superior efficiency, providing utility improvements ranging from 18% to 47% compared to prior works under identical privacy budgets ($\epsilon$). Visual analysis using Grad-CAM further confirmed that models trained with DP-Hero maintain a sharper focus on relevant objects within images, whereas standard DP-SGD models often produce blurry, unfocused feature representations.

Key Insights

  1. The "Free" Lunch of DP: The model's current state is a repository of previously protected statistical knowledge. Reusing this state to shape future noise is a highly efficient way to boost utility without increasing privacy risk.
  2. Anisotropy and Stability: Data gradients are not uniform; they have specific directions of high and low variance. By aligning noise with this anisotropy, DP-Hero stabilizes training. This prevents "model collapse" in high-noise scenarios where standard DP-SGD often fails to converge.
  3. Scalability: The framework is versatile enough to be extended to distributed settings, such as Federated Learning (via the FedFed protocol), proving that "noise wisdom" is applicable across various training architectures.

Limitations and Future Work

While DP-Hero represents a significant step forward, two primary challenges remain. First, the computational overhead of performing PCA on large parameter matrices at every iteration is non-trivial. While block-partitioning strategies can mitigate this, more lightweight guidance extraction methods are needed. Second, the current framework treats parameters as a flat matrix. Future iterations could benefit from "architecture-aware" noise construction that accounts for the structural differences between convolutional and fully connected layers.

Conclusion

DP-Hero shifts the perspective on differentially private training from uniform noise injection to intelligent, heterogeneous allocation. By learning from the knowledge already encoded in the model, we can significantly narrow the utility gap, making differentially private deep learning more viable for complex, real-world applications.

Summary updated

Abstract

Differential privacy (DP) provides a provable framework for protecting individuals by customizing a random mechanism over a privacy-sensitive dataset. Deep learning models have demonstrated privacy risks in model exposure as an established learning model unintentionally records membership-level privacy leakage. Differentially private stochastic gradient descent (DP- SGD) has been proposed to safeguard training individuals by adding random Gaussian noise to gradient updates in the backpropagation. Researchers identify that DP-SGD typically causes utility loss since the injected homogeneous noise alters the gradient updates calculated at each iteration. Namely, all elements in the gradient are contaminated regardless of their importance in updating model parameters. In this work, we argue that the utility loss mainly results from the homogeneity of injected noise. Consequently, we propose a generic differential privacy framework with heterogeneous noise (DP-Hero) by defining a heterogeneous random mechanism to abstract its property. The insight of DP-Hero is to leverage the knowledge encoded in the previously trained model to guide the subsequent allocation of noise heterogeneity, thereby leveraging the statistical perturbation and achieving enhanced utility. Atop DP-Hero, we instantiate a heterogeneous version of DP-SGD, where the noise injected into gradients is heterogeneous and guided by prior-established model parameters. We conduct comprehensive experiments to verify and explain the effectiveness of the proposed DP-Hero, showing improved training accuracy compared with state-of-the-art works. Broadly, we shed light on improving the privacy-utility space by learning the noise guidance from the pre-existing leaked knowledge encoded in the previously trained model, showing a different perspective of understanding the utility-improved DP training.
BibTeX
@misc{zheng2024rethinking,
  author = {Yu Zheng and Wenchao Zhang and Yonggang Zhang and Wei Song and Kai Zhou and Bo Han},
  title = {{Rethinking Improved Privacy-Utility Trade-off with Pre-existing Knowledge for DP Training}},
  year = {2024},
  eprint = {2409.03344},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2409.03344},
}

Loading related content...

Figure