TL;DR
This paper introduces practical methods for managing privacy budgets in machine learning, particularly for deep learning. By leveraging Rényi Differential Privacy (RDP), they develop "privacy filters" and "privacy odometers" that allow for adaptive privacy spending, leading to better accuracy and reduced privacy loss compared to existing fixed-budget approaches.
What Problem Does This Paper Address?
Training machine learning models, especially deep learning models, on sensitive data requires strong privacy guarantees. Differential Privacy (DP) is the gold standard for this, but current DP methods often rely on "composition theorems" to track the total privacy cost over a sequence of computations (like gradient updates). The challenge is that these theorems typically require a fixed privacy budget for each step, which is inflexible. This prevents algorithms from adapting their privacy spending on the fly (e.g., using less privacy for less sensitive steps) or stopping early when a desired accuracy is reached, thus wasting privacy budget. Existing methods for adaptive privacy budgets are often too complex or have impractical constants.
Methods
The core contribution is the application of Rényi Differential Privacy (RDP) to enable more practical adaptive privacy mechanisms.
- RDP: RDP is a relaxation of DP that provides a more convenient way to compose privacy guarantees. It uses a parameter $\alpha$ (the order of divergence) and allows for additive composition of privacy budgets at each order.
- Privacy Filters: The authors introduce a "privacy filter" that acts as a gatekeeper for adaptive privacy budgets. It allows computations to proceed as long as their cumulative privacy cost (tracked using RDP) doesn't exceed a predefined total privacy loss. This filter is designed to be practical and doesn't add overhead compared to non-adaptive RDP.
- Privacy Odometers: Building on privacy filters, they develop "privacy odometers." These mechanisms provide a running upper-bound on the privacy loss incurred so far, allowing for early stopping. This is crucial for scenarios where the number of required training steps is unknown, like fine-tuning. The odometer leverages RDP composition to provide tighter bounds, especially in the large-budget regime.
- Adaptive DP-SGD: They demonstrate how these filters and odometers can be integrated into DP Stochastic Gradient Descent (DP-SGD) to adapt the noise added to gradients or the batch size dynamically during training.
Datasets
- CIFAR-10: Used for evaluating adaptive privacy filters in deep learning models (ResNet9).
- ImageNet (pre-trained model): Used for fine-tuning experiments on CIFAR-10 to demonstrate privacy odometer applications.
Key Insights and Ideas
- RDP as a Unifier: RDP proves to be a powerful tool for both privacy filters and odometers, offering a more tractable way to handle adaptive privacy budgets than traditional DP composition.
- Practical Adaptivity: The proposed privacy filters and odometers are designed to be practical, with smaller constants and simpler implementations than previous adaptive DP methods. This makes them usable in real-world algorithm design.
- Early Stopping Pays Off: The privacy odometer enables significant privacy savings by allowing models to stop training early once a desired performance level is reached, rather than spending the full budget for a fixed number of epochs.
- Accuracy vs. Privacy Trade-off: Adaptive strategies can lead to better model accuracy for a fixed total privacy budget by intelligently allocating privacy where it's most needed or least impactful.
Key Results
- Improved Accuracy: Adaptive noise or batch size policies in DP-SGD on CIFAR-10 led to an average accuracy increase of 2.64% compared to a non-adaptive baseline, within a fixed total privacy loss.
- Significant Privacy Savings: In fine-tuning experiments, the privacy odometer allowed stopping training much earlier, reducing the total privacy budget spent by up to 20% (e.g., from €5.76 to €4.7 for a non-adaptive run, and even more dramatically to €1.45 when combined with an adaptive policy aiming for 80% accuracy).
- Efficiency: The privacy filter approach incurs no additional overhead compared to non-adaptive RDP composition.
Limitations and Future Work
- Complexity of RDP Orders: While RDP is powerful, choosing the optimal set of RDP orders ($\alpha$) to track can involve a trade-off between the computational cost and the tightness of the privacy bound. The paper suggests a practical set of orders for deep learning.
- Future Work: The authors hope their work will inspire broader applications, including the design of new DP algorithms, practical parameter tuning, and architecture search for DP machine learning models. They also highlight the potential for these methods in scenarios where large privacy budgets are expected.
Summary updated