Robustness Within a Fairness Budget: Parity Frontiers and Equal-Opportunity Reversal

Pablo Rivas, Jeremy FrankarXiv 2026 · arXiv

TL;DR

This paper introduces a way to integrate fairness constraints directly into adversarial training by using an explicit "fairness budget." The authors demonstrate that while we can achieve a useful balance between robustness and demographic parity, tightening the budget too much can trigger an "Equal-Opportunity Reversal," where the group originally favored by the model suddenly faces a lower true-positive rate than the other group.

What Problem Does This Paper Address?

Machine learning models often face two conflicting pressures: robustness (the ability to resist adversarial attacks) and fairness (ensuring decisions aren't biased against protected groups).

Standard adversarial training, like Projected Gradient Descent (PGD), makes models harder to fool but doesn't necessarily fix—and can sometimes worsen—existing group disparities. Conversely, traditional fairness techniques often ignore how a model might behave under attack. The authors tackle the challenge of optimizing both simultaneously. They specifically ask: How can we optimize robustness and group parity in a single program, and what happens to other fairness metrics when we do?

Methods

The authors formulate binary classification as a constrained optimization problem. They use PGD adversarial-risk minimization but add a "soft" budget for Demographic Parity (DP).

Key technical components include:

  • Squared Soft-DP Budget: Instead of a fixed penalty, they use a feasibility budget ($\tau$). If the difference in mean predicted probabilities between groups exceeds $\sqrt{\tau}$, the optimizer applies pressure to fix it.
  • Augmented Lagrangian Optimization: They use projected multiplier and one-sided penalty updates to adaptively respond to constraint violations during training.
  • Auditing vs. Prediction: In their study on the Adult Income dataset, they use the "sex" attribute for group auditing and constraint enforcement but strictly exclude it from the features used for prediction.
  • Rate Theorem: They derive a mathematical "rate theorem" that predicts exactly when a shift in demographic parity will cause a reversal in Equal Opportunity (EOpp) (the difference in true-positive rates between groups).
Figure 1: Schematic overview. A parity budget coordinates robust training with group-rate control. Curves, axis values, and group bars are illustrative, not measured results. Section 3 gives the exact updates; Section 5 reports the operating paths and EOpp reversal.

Datasets

The study uses the OpenML Adult Income dataset (a classic benchmark for algorithmic fairness). The task is to predict whether an individual's annual income exceeds $50,000. The authors focus on "sex" as the protected attribute and perturb six continuous features (like age and hours-per-week) to simulate adversarial attacks.

Key Insights and Ideas

  • Robustness is not Fairness: PGD training successfully stabilizes a model's performance under attack, but it largely leaves the "clean" fairness gaps untouched. Robustness acts as a stabilizer, not a corrector.
  • The "Frontier" of Trade-offs: By sweeping the fairness budget ($\tau$), the authors map out a "parity frontier." This allows practitioners to choose an operating point that maintains a "robust margin" (better accuracy under attack than standard models) while significantly reducing group disparity.
  • Equal-Opportunity Reversal: This is the paper's most striking finding. As you tighten the demographic parity budget, you eventually hit a point where the group with the higher true-positive rate (TPR) flips. This proves that you cannot use one fairness metric as a proxy for all others; a model that looks "fair" in terms of total positive predictions might be highly "unfair" in terms of who it correctly identifies.
  • Leveling Down: The authors observe that most of the "fairness" gains come from lowering the performance of the initially higher-performing group rather than just raising the lower group. This "leveling down" effect is a critical welfare consideration for real-world deployments.

Key Results

The experiments compared standard Empirical Risk Minimization (ERM), unconstrained PGD training, and "Fair PGD" across various attack strengths ($\varepsilon$).

  • Accuracy Gains: At an attack strength of $\varepsilon = 0.3$, PGD training improved balanced accuracy over ERM by 7.77 to 11.45 percentage points.
  • Parity Improvement: Using a representative budget of $\tau = 0.015$, the Fair PGD model cut the demographic parity difference (DPD) by roughly 50% compared to unconstrained PGD training.
  • The Robust Margin: Even with the fairness constraints, the models remained significantly more accurate under attack than standard ERM models (a margin of 3.18 to 8.34 points).
  • Metric Reversal: The authors empirically confirmed their rate theorem, showing that signed EOpp crosses zero as the DP budget tightens, as seen in the right-hand panel of the results below.
Figure 3: The tradeoff path between robustness and parity (left) and the reversal of Equal Opportunity direction (right).

Limitations and Future Work

  • Dataset Scope: The study is limited to the Adult Income dataset and a binary representation of sex. Future work is needed to see if these "frontiers" hold for non-binary or intersectional identities and other tabular datasets.
  • Feature-Space Focus: The adversarial model is a "feature-space" study, meaning it perturbs standardized numeric values. It doesn't account for semantic constraints (e.g., ensuring an attack doesn't create a logically impossible person).
  • Proxy Effects: While the protected attribute was removed from prediction, the authors acknowledge that "proxy structure" in other covariates likely remains.
  • Optimization vs. Audit: The authors emphasize that their theorem is an audit tool, not an optimization guarantee. Satisfying a smooth fairness surrogate during training does not formally guarantee the behavior of the final "hard" labels.

Summary updated

Abstract

Adversarial training protects predictive performance under attack but does not determine how decisions are distributed across protected groups. We formulate binary classification as projected gradient descent (PGD) adversarial-risk minimization under a squared soft demographic-parity (DP) budget, optimized with projected multiplier and one-sided penalty updates. On Adult Income, using the binary sex field supplied by the benchmark for group auditing but excluding it from prediction, we test logistic regression and a one-hidden-layer multilayer perceptron (MLP) across three paired seeds, four PGD strengths, and five budgets. At $\varepsilon=0.3$, PGD training improves balanced accuracy over empirical-risk minimization (ERM) by $7.77$ and $11.45$ percentage points and removes $85.7\%$ and $89.5\%$ of attack-induced growth in hard demographic-parity difference (DPD). At the representative interior value $τ=0.015$, constrained training cuts DPD relative to PGD training by $52.2\%$ and $49.2\%$, while remaining $3.18$ and $8.34$ balanced-accuracy points above ERM. Across all eight seed-averaged model-attack paths, the sampled budget path strictly orders DPD and accuracy and reverses mean signed equal-opportunity difference (EOpp). A rate theorem gives the exact true-positive-rate shift required for reversal, and the measured group rates satisfy it. Within this controlled feature-space study, robustness and parity admit useful joint operating points, but one fairness metric cannot replace a full group-rate audit.
BibTeX
@misc{rivas2026robustness,
  author = {Pablo Rivas and Jeremy Frank},
  title = {{Robustness Within a Fairness Budget: Parity Frontiers and Equal-Opportunity Reversal}},
  year = {2026},
  eprint = {2611.16031},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.16031},
}
Figure