TL;DR
Seemingly "fair" machine learning models can ironically harm marginalized groups over the long run when evaluated against downstream outcomes. ELF (Enforcing Long-term Fairness) is the first classification algorithm that provides high-confidence statistical guarantees on delayed impact (long-term well-being) using only historical observational data, bypassing the need for complex, hand-crafted analytical models or environment simulators.
What Problem Does This Paper Address?
Most fairness-aware machine learning relies on static fairness criteria evaluated at decision time—such as demographic parity or equal opportunity. However, prior research (notably Liu et al., 2018) demonstrated that satisfying static parity today can worsen inequalities tomorrow. For example, granting loans to meet an immediate acceptance quota without regard to repayment consequences can degrade the long-term credit scores and savings rates of historically disadvantaged groups.
Mitigating this "delayed impact" (DI) typically required complete, hand-engineered analytical models or simulators specifying exactly how predictions dictate future outcomes. In real-world socioeconomic settings, writing down accurate causal equations is practically impossible. The authors address an open challenge: How can we train a classifier that guarantees long-term fairness with high confidence when we only have access to logged historical data from a previously deployed system?
Methods
ELF frames classification under delayed-impact constraints through the lens of off-policy reinforcement learning within a Seldonian optimization framework:
-
Importance Sampling for Delayed Impact: ELF treats past classifier predictions as actions from a behavior policy $\beta$ and observed long-term outcomes (e.g., savings rate two years later) as delayed-impact observations ($I^\beta$). Using importance sampling, ELF reweights historical outcomes by the ratio $\frac{\pi_\theta(X, \hat{Y}^\beta)}{\beta(X, \hat{Y}^\beta)}$ to compute unbiased estimates of what the delayed impact would be under candidate policy $\pi_\theta$.
-
Split-Dataset Seldonian Architecture:
- Candidate Selection ($D_c$): An evolutionary optimizer (CMA-ES) searches for model parameters $\theta_c$ that minimize classification loss while penalizing models likely to violate delayed-impact constraints, using an inflated confidence bound.
- Fairness Test ($D_f$): On held-out data, ELF computes rigorous $(1 - \delta)$ upper confidence bounds on the delayed-impact objective $g(\theta) = \tau - \mathbb{E}[I^{\pi_\theta} \mid c(X,Y,T)]$ using concentration inequalities (Student's $t$-test or Hoeffding’s inequality).
- Safe Fallback & Consistency: If the upper bound cannot statistically verify fairness ($U > 0$), the algorithm safely returns No Solution Found (NSF) rather than deploying an untrustworthy model. The authors prove that ELF bounds the probability of returning an unfair model to at most $\delta$ (Theorem 2) and is consistent—meaning the probability of returning a valid fair model approaches 1 as sample size $n \to \infty$ (Theorem 3).
Datasets
The authors evaluate ELF on real-world welfare data sourced from the National Data Archive on Child Abuse and Neglect (NDACAN, 2021):
- AFCARS (Adoption and Foster Care Analysis and Reporting System): Youth demographic and foster-care case data.
- NYTD (National Youth in Transition Database): Educational, financial, and employment outcomes as youth transition to adulthood.
To systematically test algorithms across varying degrees of predictability, the authors synthesize a controlled delayed-impact signal governed by parameter $\alpha \in [0, 1]$: $$I_i^\psi = \alpha \hat{Y}_i^\psi + (1 - \alpha)\mathcal{N}(\mu_T, \sigma_T^2)$$ Here, $\alpha$ regulates how strongly a decision affects delayed impact versus external demographic noise.
Key Insights and Ideas
- Offline RL meets Fair Classification: Instead of simulating complex social systems, one can treat historical predictions and logged delayed metrics as bandit/RL trajectory data and apply off-policy evaluation.
- Static Fairness Actively Fails Long-Term: Models constrained only by static criteria (demographic parity, equalized odds, predictive equality) consistently produce negative delayed impacts, demonstrating that static constraints are not merely suboptimal for long-term health—they are actively misleading.
- Admitting Ignorance (NSF): Rather than gambling when data is scarce, providing an explicit NSF option allows models to maintain mathematical risk limits ($\le \delta$) regardless of sample size.
Key Results
- Guaranteed Fairness (RQ1): Across 500 trials and across all values of $\alpha$, ELF maintained a 0% failure rate, safely under the allowed threshold ($\delta_0 = \delta_1 = 10\%$). In contrast, competing baselines (Fairlearn, Fairness Constraints, quasi-Seldonian static algorithms, and logistic regression) routinely failed the long-term fairness constraint on Black youth, showing failure rates between 60% and 100%.
- Accuracy Trade-off (RQ2): With an explicit constraint requiring classification accuracy $\ge 75\%$, ELF reached 83% accuracy at $n = 1,000$ and scaled toward unconstrained baseline performance (~90%) as data increased. Baselines achieved $>90\%$ at smaller sample sizes only by deploying unfair models.
- Data Efficiency: At $n = 4,096$, ELF successfully identified and returned a certifiably fair solution in 91% of runs.
- Robustness Across Environmental Coupling (RQ3): Even when predictions had minimal influence on downstream outcomes (low $\alpha$), ELF never returned an unfair model; it simply required more data before passing the validation threshold.
Limitations and Future Work
- Full Outcome Observability: ELF requires historical records containing delayed outcomes for both accepted and rejected decisions (or alternative treatment decisions). In settings like selective college admissions where non-admitted students disappear from administrative records, ELF cannot be applied directly without unobserved-counterfactual corrections.
- Stationarity: Guarantees assume that the underlying societal data-generating distribution does not shift between historical logging and candidate deployment.
- Multi-Step Dynamics: ELF currently handles single-step predictions with delayed outcomes. Extending off-policy long-term fairness guarantees to multi-stage Markov decision processes remains open for future research.
Summary updated