TL;DR
This paper introduces a way to integrate fairness constraints directly into adversarial training by using an explicit "fairness budget." The authors demonstrate that while we can achieve a useful balance between robustness and demographic parity, tightening the budget too much can trigger an "Equal-Opportunity Reversal," where the group originally favored by the model suddenly faces a lower true-positive rate than the other group.
What Problem Does This Paper Address?
Machine learning models often face two conflicting pressures: robustness (the ability to resist adversarial attacks) and fairness (ensuring decisions aren't biased against protected groups).
Standard adversarial training, like Projected Gradient Descent (PGD), makes models harder to fool but doesn't necessarily fix—and can sometimes worsen—existing group disparities. Conversely, traditional fairness techniques often ignore how a model might behave under attack. The authors tackle the challenge of optimizing both simultaneously. They specifically ask: How can we optimize robustness and group parity in a single program, and what happens to other fairness metrics when we do?
Methods
The authors formulate binary classification as a constrained optimization problem. They use PGD adversarial-risk minimization but add a "soft" budget for Demographic Parity (DP).
Key technical components include:
- Squared Soft-DP Budget: Instead of a fixed penalty, they use a feasibility budget ($\tau$). If the difference in mean predicted probabilities between groups exceeds $\sqrt{\tau}$, the optimizer applies pressure to fix it.
- Augmented Lagrangian Optimization: They use projected multiplier and one-sided penalty updates to adaptively respond to constraint violations during training.
- Auditing vs. Prediction: In their study on the Adult Income dataset, they use the "sex" attribute for group auditing and constraint enforcement but strictly exclude it from the features used for prediction.
- Rate Theorem: They derive a mathematical "rate theorem" that predicts exactly when a shift in demographic parity will cause a reversal in Equal Opportunity (EOpp) (the difference in true-positive rates between groups).
Datasets
The study uses the OpenML Adult Income dataset (a classic benchmark for algorithmic fairness). The task is to predict whether an individual's annual income exceeds $50,000. The authors focus on "sex" as the protected attribute and perturb six continuous features (like age and hours-per-week) to simulate adversarial attacks.
Key Insights and Ideas
- Robustness is not Fairness: PGD training successfully stabilizes a model's performance under attack, but it largely leaves the "clean" fairness gaps untouched. Robustness acts as a stabilizer, not a corrector.
- The "Frontier" of Trade-offs: By sweeping the fairness budget ($\tau$), the authors map out a "parity frontier." This allows practitioners to choose an operating point that maintains a "robust margin" (better accuracy under attack than standard models) while significantly reducing group disparity.
- Equal-Opportunity Reversal: This is the paper's most striking finding. As you tighten the demographic parity budget, you eventually hit a point where the group with the higher true-positive rate (TPR) flips. This proves that you cannot use one fairness metric as a proxy for all others; a model that looks "fair" in terms of total positive predictions might be highly "unfair" in terms of who it correctly identifies.
- Leveling Down: The authors observe that most of the "fairness" gains come from lowering the performance of the initially higher-performing group rather than just raising the lower group. This "leveling down" effect is a critical welfare consideration for real-world deployments.
Key Results
The experiments compared standard Empirical Risk Minimization (ERM), unconstrained PGD training, and "Fair PGD" across various attack strengths ($\varepsilon$).
- Accuracy Gains: At an attack strength of $\varepsilon = 0.3$, PGD training improved balanced accuracy over ERM by 7.77 to 11.45 percentage points.
- Parity Improvement: Using a representative budget of $\tau = 0.015$, the Fair PGD model cut the demographic parity difference (DPD) by roughly 50% compared to unconstrained PGD training.
- The Robust Margin: Even with the fairness constraints, the models remained significantly more accurate under attack than standard ERM models (a margin of 3.18 to 8.34 points).
- Metric Reversal: The authors empirically confirmed their rate theorem, showing that signed EOpp crosses zero as the DP budget tightens, as seen in the right-hand panel of the results below.
Limitations and Future Work
- Dataset Scope: The study is limited to the Adult Income dataset and a binary representation of sex. Future work is needed to see if these "frontiers" hold for non-binary or intersectional identities and other tabular datasets.
- Feature-Space Focus: The adversarial model is a "feature-space" study, meaning it perturbs standardized numeric values. It doesn't account for semantic constraints (e.g., ensuring an attack doesn't create a logically impossible person).
- Proxy Effects: While the protected attribute was removed from prediction, the authors acknowledge that "proxy structure" in other covariates likely remains.
- Optimization vs. Audit: The authors emphasize that their theorem is an audit tool, not an optimization guarantee. Satisfying a smooth fairness surrogate during training does not formally guarantee the behavior of the final "hard" labels.
Summary updated