Privacy Risk for anisotropic Langevin dynamics using relative entropy bounds

Anastasia Borovykh, Nikolas Kantas, Panos Parpas, Greg PavliotisarXiv 2023 · OpenReview

Abstract

The privacy preserving properties of Langevin dynamics with additive isotropic noise have been extensively studied. However, the isotropic noise assumption is very restrictive: (a) when adding noise to existing learning algorithms to preserve privacy and maintain the best possible accuracy one should take into account the relative magnitude of the outputs and their correlations; (b) popular algorithms such as stochastic gradient descent (and their continuous time limits) appear to possess anisotropic covariance properties. To study the privacy risks for the anisotropic noise case, one requires general results on the relative entropy between the laws of two Stochastic Differential Equations with different drifts and diffusion coefficients. Our main contribution is to establish such a bound using stability estimates for solutions to the Fokker-Planck equations via functional inequalities. With additional assumptions, the relative entropy bound implies an $(\epsilon,\delta)$-differential privacy bound or translates to bounds on the membership inference attack success and we show how anisotropic noise can lead to better privacy-accuracy trade-offs. Finally, the benefits of anisotropic noise are illustrated using numerical results in quadratic loss and neural network setups.

Introduction

In the training of privacy-preserving machine learning models, noise injection is a standard technique to prevent the memorization of individual training samples. Traditionally, theoretical guarantees for Differential Privacy (DP) have relied on the assumption of isotropic noise—noise that is uniform in all directions. However, this assumption is often at odds with the reality of optimization. Stochastic Gradient Descent (SGD) naturally exhibits anisotropic noise because the gradient variance is tied to the local geometry of the loss landscape.

Furthermore, forcing noise to be isotropic can be detrimental to model performance. This paper introduces a theoretical framework to quantify privacy risks in non-reversible, anisotropic systems, demonstrating that varying noise magnitude across different directions can optimize the trade-off between privacy and accuracy.

Theoretical Framework

The authors model the training process as an Itô Stochastic Differential Equation (SDE): $$dx_t = -\nabla f(x_t)dt + \Sigma(x_t)dW_t$$ where $\Sigma$ represents the noise structure or diffusion coefficient.

The core contribution is Theorem 2.2, which provides a bound on the time-derivative of the Kullback-Leibler (KL) divergence between the laws of two SDEs. Unlike standard approaches that use Girsanov transforms (which are limited when noise structures differ), this work utilizes stability estimates of the Fokker-Planck equation. By applying Log-Sobolev Inequalities (LSI), these entropy bounds are converted into concrete $(\epsilon, \delta)$-DP guarantees and membership inference attack success rates.

The Advantage of Anisotropy

A key insight of this research is that privacy risk and model accuracy often respond differently to noise in different parameter directions.

Privacy risk landscape showing sensitivity to noise in specific directions.

As shown in the heatmaps, privacy risk may be highly sensitive to noise in one direction (e.g., the 'x' axis) while remaining relatively unaffected by noise in another. Conversely, model accuracy typically degrades based on the total magnitude of noise added across all dimensions, resulting in radial contours.

Accuracy landscape showing how total noise magnitude affects model performance.

By identifying "critical directions"—where gradients of different datasets diverge most—practitioners can inject noise primarily in those directions. This "hides" individual data points effectively while preserving the signal in directions critical for accuracy, leading to a superior Pareto front compared to uniform noise injection.

Experimental Results

The theoretical findings were validated across several setups:

  • Synthetic Quadratic Loss: Demonstrated the link between the condition number of the loss landscape and privacy.
  • FashionMNIST & CIFAR10: In neural network experiments, anisotropic noise consistently achieved lower "worst-case loss" for the same privacy level ($\delta$). For CIFAR10, isotropic noise often hindered convergence, whereas anisotropic noise (scaled by gradient magnitude) allowed for successful training with strong privacy bounds.

The analysis also confirmed that larger batch sizes in SGD increase KL divergence and privacy risk, providing a formal mathematical basis for the heuristic that smaller batches are more private.

Conclusion and Future Work

This work provides a robust mathematical foundation for using anisotropic noise in Langevin dynamics. While the tightest bounds currently rely on the Log-Sobolev Inequality—which is challenging to verify for non-convex neural networks—the practical implications are clear: noise should be tailored to the loss landscape. Future research will focus on identifying optimal parameter directions to further enhance privacy-preserving algorithms.

Summary updated

Abstract

The privacy preserving properties of Langevin dynamics with additive isotropic noise have been extensively studied. However, the isotropic noise assumption is very restrictive: (a) when adding noise to existing learning algorithms to preserve privacy and maintain the best possible accuracy one should take into account the relative magnitude of the outputs and their correlations; (b) popular algorithms such as stochastic gradient descent (and their continuous time limits) appear to possess anisotropic covariance properties. To study the privacy risks for the anisotropic noise case, one requires general results on the relative entropy between the laws of two Stochastic Differential Equations with different drifts and diffusion coefficients. Our main contribution is to establish such a bound using stability estimates for solutions to the Fokker-Planck equations via functional inequalities. With additional assumptions, the relative entropy bound implies an $(\epsilon,\delta)$-differential privacy bound or translates to bounds on the membership inference attack success and we show how anisotropic noise can lead to better privacy-accuracy trade-offs. Finally, the benefits of anisotropic noise are illustrated using numerical results in quadratic loss and neural network setups.
BibTeX
@misc{borovykh2023privacy,
  author = {Anastasia Borovykh and Nikolas Kantas and Panos Parpas and Greg Pavliotis},
  title = {{Privacy Risk for anisotropic Langevin dynamics using relative entropy bounds}},
  year = {2023},
  eprint = {2302.00766},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2302.00766},
}
Figure