Practical Privacy Filters and Odometers with Rényi Differential Privacy and Applications to Differentially Private Deep Learning

Mathias LécuyerarXiv 2021 · OpenReview

TL;DR

This paper introduces practical methods for managing privacy budgets in machine learning, particularly for deep learning. By leveraging Rényi Differential Privacy (RDP), they develop "privacy filters" and "privacy odometers" that allow for adaptive privacy spending, leading to better accuracy and reduced privacy loss compared to existing fixed-budget approaches.

What Problem Does This Paper Address?

Training machine learning models, especially deep learning models, on sensitive data requires strong privacy guarantees. Differential Privacy (DP) is the gold standard for this, but current DP methods often rely on "composition theorems" to track the total privacy cost over a sequence of computations (like gradient updates). The challenge is that these theorems typically require a fixed privacy budget for each step, which is inflexible. This prevents algorithms from adapting their privacy spending on the fly (e.g., using less privacy for less sensitive steps) or stopping early when a desired accuracy is reached, thus wasting privacy budget. Existing methods for adaptive privacy budgets are often too complex or have impractical constants.

Methods

The core contribution is the application of Rényi Differential Privacy (RDP) to enable more practical adaptive privacy mechanisms.

  • RDP: RDP is a relaxation of DP that provides a more convenient way to compose privacy guarantees. It uses a parameter $\alpha$ (the order of divergence) and allows for additive composition of privacy budgets at each order.
  • Privacy Filters: The authors introduce a "privacy filter" that acts as a gatekeeper for adaptive privacy budgets. It allows computations to proceed as long as their cumulative privacy cost (tracked using RDP) doesn't exceed a predefined total privacy loss. This filter is designed to be practical and doesn't add overhead compared to non-adaptive RDP.
  • Privacy Odometers: Building on privacy filters, they develop "privacy odometers." These mechanisms provide a running upper-bound on the privacy loss incurred so far, allowing for early stopping. This is crucial for scenarios where the number of required training steps is unknown, like fine-tuning. The odometer leverages RDP composition to provide tighter bounds, especially in the large-budget regime.
  • Adaptive DP-SGD: They demonstrate how these filters and odometers can be integrated into DP Stochastic Gradient Descent (DP-SGD) to adapt the noise added to gradients or the batch size dynamically during training.

Datasets

  • CIFAR-10: Used for evaluating adaptive privacy filters in deep learning models (ResNet9).
  • ImageNet (pre-trained model): Used for fine-tuning experiments on CIFAR-10 to demonstrate privacy odometer applications.

Key Insights and Ideas

  • RDP as a Unifier: RDP proves to be a powerful tool for both privacy filters and odometers, offering a more tractable way to handle adaptive privacy budgets than traditional DP composition.
  • Practical Adaptivity: The proposed privacy filters and odometers are designed to be practical, with smaller constants and simpler implementations than previous adaptive DP methods. This makes them usable in real-world algorithm design.
  • Early Stopping Pays Off: The privacy odometer enables significant privacy savings by allowing models to stop training early once a desired performance level is reached, rather than spending the full budget for a fixed number of epochs.
  • Accuracy vs. Privacy Trade-off: Adaptive strategies can lead to better model accuracy for a fixed total privacy budget by intelligently allocating privacy where it's most needed or least impactful.

Key Results

  • Improved Accuracy: Adaptive noise or batch size policies in DP-SGD on CIFAR-10 led to an average accuracy increase of 2.64% compared to a non-adaptive baseline, within a fixed total privacy loss.
  • Significant Privacy Savings: In fine-tuning experiments, the privacy odometer allowed stopping training much earlier, reducing the total privacy budget spent by up to 20% (e.g., from €5.76 to €4.7 for a non-adaptive run, and even more dramatically to €1.45 when combined with an adaptive policy aiming for 80% accuracy).
  • Efficiency: The privacy filter approach incurs no additional overhead compared to non-adaptive RDP composition.

Limitations and Future Work

  • Complexity of RDP Orders: While RDP is powerful, choosing the optimal set of RDP orders ($\alpha$) to track can involve a trade-off between the computational cost and the tightness of the privacy bound. The paper suggests a practical set of orders for deep learning.
  • Future Work: The authors hope their work will inspire broader applications, including the design of new DP algorithms, practical parameter tuning, and architecture search for DP machine learning models. They also highlight the potential for these methods in scenarios where large privacy budgets are expected.

Summary updated

Abstract

Differential Privacy (DP) is the leading approach to privacy preserving deep learning. As such, there are multiple efforts to provide drop-in integration of DP into popular frameworks. These efforts, which add noise to each gradient computation to make it DP, rely on composition theorems to bound the total privacy loss incurred over this sequence of DP computations. However, existing composition theorems present a tension between efficiency and flexibility. Most theorems require all computations in the sequence to have a predefined DP parameter, called the privacy budget. This prevents the design of training algorithms that adapt the privacy budget on the fly, or that terminate early to reduce the total privacy loss. Alternatively, the few existing composition results for adaptive privacy budgets provide complex bounds on the privacy loss, with constants too large to be practical. In this paper, we study DP composition under adaptive privacy budgets through the lens of R\'enyi Differential Privacy, proving a simpler composition theorem with smaller constants, making it practical enough to use in algorithm design. We demonstrate two applications of this theorem for DP deep learning: adapting the noise or batch size online to improve a model's accuracy within a fixed total privacy loss, and stopping early when fine-tuning a model to reduce total privacy loss.
BibTeX
@misc{lecuyer2021practical,
  author = {Mathias Lécuyer},
  title = {{Practical Privacy Filters and Odometers with Rényi Differential Privacy and Applications to Differentially Private Deep Learning}},
  year = {2021},
  eprint = {2103.01379},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2103.01379},
}
Figure