NeuralDP Differentially private neural networks by design

Moritz Knolle, Dmitrii Usynin, Alexander Ziller, Marcus R. Makowski, Daniel Rueckert, Georgios KaissisarXiv 2021 · OpenReview

TL;DR

This paper introduces NeuralDP, a novel method for training differentially private deep neural networks by privatizing intermediate layer activations instead of gradients. This approach offers a better privacy-utility trade-off compared to the standard DP-SGD method, especially in high-privacy settings.

What Problem Does This Paper Address?

Deep neural networks (DNNs) have revolutionized many fields, but their application is often hindered by the sensitive nature of the data required for training. Centralizing and aggregating such data raises significant privacy concerns. Differential Privacy (DP) offers a mathematical guarantee of privacy, but applying it to DNN training has been challenging. The dominant method, DP-SGD, privatizes gradients by clipping them and adding noise. While effective, this can significantly degrade model performance (utility) and introduce biases in the training process. This paper aims to find a more efficient way to achieve DP in DNNs, improving the balance between privacy and model accuracy.

Methods

The core of NeuralDP lies in privatizing the activations of a specific layer within the neural network, rather than the gradients. This is achieved by:

  1. Enforcing Lipschitz Continuity: The paper leverages the property that if a layer is Lipschitz continuous, its sensitivity (how much its output can change with a small change in input) is bounded by its Lipschitz constant. This is enforced using spectral normalization on the layer's weight matrix. The authors also use the GroupSort activation function, which is known to be Lipschitz continuous.
  2. Applying the Gaussian Mechanism: Once the Lipschitz constant of a layer is fixed, the Gaussian mechanism is applied to its activations. This involves adding Gaussian noise scaled by the layer's Lipschitz constant and a privacy multiplier.
  3. Leveraging Post-Processing: A key insight is that due to the "post-processing" property of differential privacy, if a single layer's output is differentially private, the entire network's output becomes differentially private. This means the rest of the network can process these privatized activations without compromising the overall privacy guarantee.

This contrasts with DP-SGD, which directly clips and adds noise to gradients at each training step.

Datasets

The paper evaluates NeuralDP on two datasets:

  • MNIST: A standard benchmark dataset for handwritten digit recognition.
  • Pediatric Pneumonia Dataset (PPD): A dataset for medical imaging, highlighting the application of DP in sensitive domains.

Key Insights and Ideas

  • Privatizing Activations vs. Gradients: The paper demonstrates that privatizing intermediate activations can be more effective than privatizing gradients, leading to better utility for a given privacy budget.
  • The Power of Lipschitz Continuity: Enforcing Lipschitz continuity on a specific layer is crucial. It bounds the sensitivity of that layer's output, allowing for effective application of the Gaussian mechanism.
  • Post-Processing is Key: The theoretical foundation that privatizing a single component (a layer's activation) makes the entire output DP is a powerful simplification.
  • Resilience of Activations: The authors suggest that intermediate activations might be more resilient to noise perturbation than gradients, contributing to better utility.

Key Results

  • Improved Privacy-Utility Trade-off: Experiments on MNIST and PPD show that NeuralDP offers substantially improved privacy-utility trade-offs compared to DP-SGD. For example, on the PPD dataset with a privacy budget of $\epsilon=6.0$, NeuralDP achieved an accuracy of 0.854 and ROC-AUC of 0.963, while DP-SGD achieved 0.85 and 0.89 respectively.
  • Comparable Performance to Non-Private SGD: In some cases, NeuralDP's accuracy is close to that of non-private SGD, indicating minimal utility loss.
  • Optimal Layer Placement: Placing the NeuralDP layer at an intermediate position (dense or convolutional) generally yields the best performance.
  • Sensitivity to Depth: While NeuralDP performs well with increasing model width, it shows convergence issues with very deep convolutional networks (more than eight layers).

Limitations and Future Work

  • Recurrent Architectures: The current NeuralDP method has not been trivially extended to recurrent neural networks.
  • Limited Datasets: Evaluation was primarily on MNIST and PPD.
  • Reconstruction Attacks: Further research is suggested to explore reconstruction abilities compared to DP-SGD and to conduct ablation studies on the orthonormality constraints.
  • Convergence for Deep Networks: Addressing the convergence issues for very deep convolutional networks is a potential area for future work, possibly by incorporating residual connections.
  • Exploration of Other DP Mechanisms: While the Gaussian mechanism is used, exploring other DP mechanisms could be beneficial.

Summary updated

Abstract

The application of differential privacy to the training of deep neural networks holds the promise of allowing large-scale (decentralized) use of sensitive data while providing rigorous privacy guarantees to the individual. The predominant approach to differentially private training of neural networks is DP-SGD, which relies on norm-based gradient clipping as a method for bounding sensitivity, followed by the addition of appropriately calibrated Gaussian noise. In this work we propose NeuralDP, a technique for privatising activations of some layer within a neural network, which by the post-processing properties of differential privacy yields a differentially private network. We experimentally demonstrate on two datasets (MNIST and Pediatric Pneumonia Dataset (PPD)) that our method offers substantially improved privacy-utility trade-offs compared to DP-SGD.
BibTeX
@misc{knolle2021neuraldp,
  author = {Moritz Knolle and Dmitrii Usynin and Alexander Ziller and Marcus R. Makowski and Daniel Rueckert and Georgios Kaissis},
  title = {{NeuralDP Differentially private neural networks by design}},
  year = {2021},
  eprint = {2107.14582},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2107.14582},
}
Figure