Scalable Differential Privacy with Certified Robustness in Adversarial Learning

Hai Phan, My Thai, Han Hu, Ruoming Jin, Tong Sun, Dejing DouOpenReview 2021 · OpenReview

TL;DR

This paper introduces a novel "Stochastic Batch" (StoBatch) mechanism that enables scalable deep learning models to be both differentially private (protecting training data) and certified robust against adversarial attacks. By combining these two crucial security properties, StoBatch addresses a significant gap in current machine learning security, offering a more comprehensive defense for deployed models.

What Problem Does This Paper Address?

Machine learning models, especially deep neural networks (DNNs), face two major security threats:

  1. Privacy Leakage: Deployed models can inadvertently reveal sensitive information about the private data they were trained on. Differential Privacy (DP) is a rigorous mathematical framework to prevent this.
  2. Adversarial Attacks: Malicious actors can craft subtle perturbations to input data (adversarial examples) that cause models to misclassify them, even though the perturbations are imperceptible to humans. Adversarial training and certified robustness aim to mitigate this.

The core challenge is that existing methods for achieving DP and adversarial robustness are often at odds. DP methods can degrade model utility and robustness, while adversarial training methods typically don't offer privacy guarantees for the training data. Combining them naively doesn't work because adversarial training itself can introduce new privacy risks by exposing information about the training data through the adversarial examples generated. This paper aims to bridge this gap by developing a single, scalable mechanism that provides both DP and certified robustness.

Methods

The authors propose the Stochastic Batch (StoBatch) mechanism, which integrates DP and adversarial learning in a novel way. Key components include:

  • DP Feature Representation Learning: Noise is injected into both input and latent spaces of a DNN's feature extractor (an auto-encoder component) to ensure DP in learning model parameters. This is achieved by applying a functional mechanism to a polynomial approximation of the reconstruction loss.
  • DP Adversarial Learning: The mechanism crafts "DP adversarial examples" using perturbed benign data. An original adversarial objective function is designed to tighten the model's global sensitivity, reducing the amount of noise needed for DP.
  • Certified Robustness: By leveraging DP's sequential composition theory, the authors derive a generalized robustness bound that considers robustness in both input and latent spaces, going beyond previous methods that only considered one.
  • Stochastic Batch Training: To overcome the scalability limitations of traditional iterative batch-by-batch DP training, StoBatch uses a distributed approach. Disjoint and fixed batches are distributed to local trainers, allowing for efficient computation of adversarial examples across multiple batches simultaneously. This enables training on large DNNs and datasets.

Datasets

The proposed method was evaluated on standard benchmark datasets:

  • MNIST: Handwritten digits (28x28 grayscale images).
  • CIFAR-10: Color images across 10 classes (32x32 images).
  • Tiny ImageNet: A larger dataset with 200 classes (64x64 images), using a ResNet18 model.

Key Insights and Ideas

  • Synergy between DP and Robustness: The paper demonstrates that DP noise and adversarial robustness noise can be correlated and even leveraged to enhance each other, rather than being mutually exclusive.
  • Input and Latent Space Randomization: Randomizing both input and latent spaces, and composing their robustness bounds, leads to stronger certified robustness guarantees than methods focusing on only one space.
  • Scalability through Distributed Training: The novel stochastic batch training approach is crucial for applying DP and adversarial learning to large-scale DNNs and datasets, overcoming a major bottleneck in prior work.
  • Trade-off Management: The proposed adversarial objective function and sensitivity analysis help to better manage the inherent trade-off between model utility, privacy loss, and robustness.

Key Results

  • Improved Accuracy: StoBatch consistently outperforms baseline methods (DP-SGD, AdLM, SecureSGD, SecureSGD-AGM, PixelDP) in terms of both conventional and certified accuracy across MNIST, CIFAR-10, and Tiny ImageNet datasets, especially under strong DP guarantees and adversarial attacks.
  • Robustness to Attacks: StoBatch shows significantly better resilience to adversarial attacks, maintaining higher accuracy as attack sizes (µα) increase, compared to other methods.
  • Tight DP Guarantees: The mechanism achieves strong DP protection (e.g., ε=0.2) with only a small degradation in conventional accuracy, unlike baselines that suffer substantial drops.
  • Scalability: The distributed StoBatch training significantly reduces training time for large models and datasets (e.g., ~3 days vs. >30 days for iterative batch-by-batch training on CIFAR-10 with strong iterative attacks).
  • Certified Accuracy: StoBatch achieves competitive certified accuracies, particularly against more aggressive attacks where other methods falter.

Limitations and Future Work

  • Limitations:
    • The paper notes that while StoBatch offers significant improvements, achieving high accuracy under very strong DP guarantees (small ε) and aggressive adversarial attacks remains challenging.
  • The current implementation focuses on distributed training; federated learning scenarios are reserved for future exploration.
  • Future Work:
    • Testing algorithms and models on the Baidu Fedcube platform.
  • Evaluating robustness bounds against synergistic attacks, which combine adversarial examples with other threats like Trojans.
  • Exploring federated learning extensions of the StoBatch mechanism.

Summary updated

Abstract

In this paper, we aim to develop a scalable algorithm to preserve differential privacy (DP) in adversarial learning for deep neural networks (DNNs), with certified robustness to adversarial examples. By leveraging the sequential composition theory in DP, we randomize both input and latent spaces to strengthen our certified robustness bounds. To address the trade-off among model utility, privacy loss, and robustness, we design an original adversarial objective function based on the post-processing property in DP to tighten the sensitivity of our model. A new stochastic batch training is proposed to apply our mechanism on large DNNs and datasets by bypassing the vanilla iterative batch-by-batch training in DP DNNs. An end-to-end theoretical analysis and evaluations show that our mechanism notably improves the robustness and scalability of DP DNNs.
BibTeX
@misc{phan2021scalable,
  author = {Hai Phan and My Thai and Han Hu and Ruoming Jin and Tong Sun and Dejing Dou},
  title = {{Scalable Differential Privacy with Certified Robustness in Adversarial Learning}},
  year = {2021},
  url = {https://openreview.net/forum?id=P35oipfsgIz},
  howpublished = {OpenReview},
}
Figure