DP-SGD vs PATE: Which Has Less Disparate Impact on Model Accuracy?

Archit Uniyal, Rakshit Naidu, Sasikanth Kotti, Sahib Singh, Patrik Joslin Kenfack, Fatemehsadat Mireshghallah, Andrew Trask

Abstract

Differential Privacy (DP) is essential for protecting individual data, but it often disproportionately lowers the accuracy of minority groups. This paper compares the industry-standard DP-SGD with the PATE mechanism. Findings demonstrate that PATE significantly reduces "disparate impact," maintaining higher utility for under-represented groups and offering a more equitable path for private machine learning.

Introduction

As machine learning models increasingly handle sensitive personal data, Differential Privacy (DP) has become a critical standard for ensuring individual privacy. By adding noise to the training process, DP guarantees that no single individual's data can be reverse-engineered from the final model. However, this protection comes with a hidden cost: "disparate impact."

Recent research indicates that the noise introduced by DP training does not affect all data points equally. Specifically, DP-SGD—the most common private training method—tends to drastically reduce accuracy on under-represented or minority sub-groups compared to well-represented ones. This creates a fairness crisis in high-stakes applications like hiring or insurance. This study investigates whether this bias is inherent to all DP methods or if alternative architectures, specifically Private Aggregation of Teacher Ensembles (PATE), can provide a more equitable solution.

Methodology

The researchers evaluated two primary frameworks for private deep learning:

  1. DP-SGD (Differentially Private Stochastic Gradient Descent): Gradients are clipped and noise is added directly to the model's weights during every training step.
  2. PATE (Private Aggregation of Teacher Ensembles): Sensitive data is split into disjoint sets, each used to train a "teacher" model. These teachers then vote on labels for a separate, non-sensitive public dataset to train a "student" model. Noise is only added to the aggregate vote count.

To simulate real-world imbalances, the researchers used modified versions of the MNIST and SVHN datasets. In both cases, class "8" was intentionally "starved," reduced to a 1:10 ratio compared to other classes (e.g., only 500 samples in MNIST).

Why PATE Mitigates Disparate Impact

The study's core finding is that PATE is significantly more "minority-friendly" than DP-SGD. The authors hypothesize several reasons for this:

  • Signal vs. Noise: DP-SGD adds noise to every gradient update, which can easily "drown out" the already weak signals from minority samples. In contrast, PATE teachers are trained on raw data without noise; the privacy-preserving noise is only introduced during the voting stage, preserving minority features more effectively.
  • Ensemble Robustness: Because PATE teachers are trained on different data subsets, they learn diverse patterns. The ensemble voting mechanism helps cancel out individual biases, leading to a more robust final student model.
  • Teacher Scaling: The researchers found a "sweet spot" for fairness—using 10 teachers provided the best balance between robust voting and sufficient data per teacher to learn useful features.

Key Results

Experimental results on MNIST showed that PATE maintained much higher utility for the minority class across various privacy budgets (ε). While DP-SGD's performance on the minority class fluctuated and dropped significantly as privacy demands increased, PATE remained stable.

Comparison of test accuracy across classes for DP-SGD and PATE at different privacy budgets (ε).

As shown in the figure above, PATE consistently outperforms DP-SGD on the minority class (Class 8). On MNIST, PATE's accuracy for the minority group was nearly double that of DP-SGD for the same privacy budget. Furthermore, PATE exhibited lower standard deviation across runs, indicating it is not only fairer but also more predictable and reliable. While both models struggled with the more complex SVHN dataset, PATE remained significantly more robust against the high-variance failures seen in DP-SGD.

Conclusion and Future Work

PATE offers a promising alternative for fair, private machine learning, particularly in imbalanced datasets. Its primary limitation is the requirement for public, unlabeled data to train the student model—a resource that is often unavailable in sensitive fields like medical imaging. Future research should focus on achieving PATE-level fairness in settings without public data or developing techniques to make DP-SGD's noise less destructive to minority signals.

Summary updated

Abstract

Recent advances in differentially private deep learning have demonstrated that application of differential privacy, specifically the DP-SGD algorithm, has a disparate impact on different sub-groups in the population, which leads to a significantly high drop-in model utility for sub-populations that are under-represented (minorities), compared to well-represented ones. In this work, we aim to compare PATE, another mechanism for training deep learning models using differential privacy, with DP-SGD in terms of fairness. We show that PATE does have a disparate impact too, however, it is much less severe than DP-SGD. We draw insights from this observation on what might be promising directions in achieving better fairness-privacy trade-offs.
BibTeX
@misc{uniyal2021dp,
  author = {Archit Uniyal and Rakshit Naidu and Sasikanth Kotti and Sahib Singh and Patrik Joslin Kenfack and Fatemehsadat Mireshghallah and Andrew Trask},
  title = {{DP-SGD vs PATE: Which Has Less Disparate Impact on Model Accuracy?}},
  year = {2021},
  eprint = {2106.12576},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2106.12576},
}
Figure