Exploring the Unfairness of DP-SGD Across Settings

Frederik Noe, Rasmus Herskind, Anders SøgaardarXiv 2022 · arXiv

Introduction

In the pursuit of "Responsible AI," researchers often treat privacy and fairness as independent objectives. However, recent evidence suggests a fundamental tension between the two. Differential Privacy (DP) techniques, such as Differentially Private Stochastic Gradient Descent (DP-SGD), protect individual data points by adding noise and clipping gradients. Unfortunately, this process often disproportionately impacts minority subgroups whose rare features are more likely to be suppressed as outliers. This paper quantifies this trade-off, establishing a mathematical relationship between the privacy budget ($\epsilon$) and fairness degradation.

Methodology

The study evaluates the impact of DP-SGD across three distinct machine learning paradigms:

  1. Linear Classification: Logistic regression models trained with DP-SGD.
  2. Robust Deep Learning (Group-DRO): Utilizing Bertweet and DistilBert architectures, Group Distributionally Robust Optimization (Group-DRO) is employed to minimize "worst-case" loss across demographic groups, theoretically providing a buffer against group disparities.
  3. Dimensionality Reduction (DP-PCA): Private data representations are generated via the IBM Differential Privacy Library before being passed to a standard classifier.

Fairness is primarily quantified using $\delta$-Unequal Risk, which measures the maximum error rate disparity between any two demographic groups, and a Modified p%-rule, which assesses how much model predictions diverge from actual group priors.

Dataset and Baseline Disparities

The research utilizes the CivilComments dataset, a large-scale toxicity detection benchmark. Comments are annotated for both toxicity and demographic identities (e.g., LGBTQ, religious groups, race), allowing for granular tracking of accuracy across protected groups. Baseline non-private models already exhibit significant performance gaps.

Baseline F1-scores across demographic groups in the CivilComments dataset.

As shown in Figure 1, toxicity detection performance varies significantly by group, with minority religious groups often seeing the lowest F1-scores, while "Black" and "White" identifiers typically see the highest.

Results: The Logarithmic Law of Unfairness

The central finding of this work is the Logarithmic Law of Unfairness: both accuracy and fairness metrics scale logarithmically with the privacy budget $\epsilon$. As privacy protections are tightened (decreasing $\epsilon$), fairness degrades rapidly and non-linearly.

Logarithmic relationship between privacy budget (epsilon), accuracy, and fairness metrics across different settings.

Crucially, the study finds that fairness-aware methods like Group-DRO are not a "silver bullet." Despite being designed to protect the worst-performing groups, Group-DRO follows the same negative logarithmic trend as standard models. Furthermore, while DP-PCA initially appeared to maintain fairness better, subsequent analysis revealed significant privacy leakage, where sensitive information could still be recovered from the "private" representations by a simple feed-forward network.

Discussion and Limitations

The results underscore a stark reality: increasing privacy via DP-SGD consistently exacerbates existing inequalities. The logarithmic nature of this trade-off means that even moderate privacy requirements can lead to substantial fairness costs.

Limitations include the specific focus on the CivilComments toxicity task and the observed privacy failures in the DP-PCA implementation, which was only approximate. Future research must explore alternative optimization strategies, perhaps utilizing second-order gradient information, to find a more equitable middle ground between privacy and fairness.

Summary updated

BibTeX
@misc{noe2022exploring,
  author = {Frederik Noe and Rasmus Herskind and Anders Søgaard},
  title = {{Exploring the Unfairness of DP-SGD Across Settings}},
  year = {2022},
  eprint = {2202.12058},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2202.12058},
}
Figure