Enforcing Delayed-Impact Fairness Guarantees

Aline Weber, Blossom Metevier, Yuriy Brun, Philip S. Thomas, Bruno Castro da Silva

TL;DR

Seemingly "fair" machine learning models can ironically harm marginalized groups over the long run when evaluated against downstream outcomes. ELF (Enforcing Long-term Fairness) is the first classification algorithm that provides high-confidence statistical guarantees on delayed impact (long-term well-being) using only historical observational data, bypassing the need for complex, hand-crafted analytical models or environment simulators.

What Problem Does This Paper Address?

Most fairness-aware machine learning relies on static fairness criteria evaluated at decision time—such as demographic parity or equal opportunity. However, prior research (notably Liu et al., 2018) demonstrated that satisfying static parity today can worsen inequalities tomorrow. For example, granting loans to meet an immediate acceptance quota without regard to repayment consequences can degrade the long-term credit scores and savings rates of historically disadvantaged groups.

Mitigating this "delayed impact" (DI) typically required complete, hand-engineered analytical models or simulators specifying exactly how predictions dictate future outcomes. In real-world socioeconomic settings, writing down accurate causal equations is practically impossible. The authors address an open challenge: How can we train a classifier that guarantees long-term fairness with high confidence when we only have access to logged historical data from a previously deployed system?

Methods

ELF frames classification under delayed-impact constraints through the lens of off-policy reinforcement learning within a Seldonian optimization framework:

  1. Importance Sampling for Delayed Impact: ELF treats past classifier predictions as actions from a behavior policy $\beta$ and observed long-term outcomes (e.g., savings rate two years later) as delayed-impact observations ($I^\beta$). Using importance sampling, ELF reweights historical outcomes by the ratio $\frac{\pi_\theta(X, \hat{Y}^\beta)}{\beta(X, \hat{Y}^\beta)}$ to compute unbiased estimates of what the delayed impact would be under candidate policy $\pi_\theta$.

  2. Split-Dataset Seldonian Architecture:

  3. Candidate Selection ($D_c$): An evolutionary optimizer (CMA-ES) searches for model parameters $\theta_c$ that minimize classification loss while penalizing models likely to violate delayed-impact constraints, using an inflated confidence bound.
  4. Fairness Test ($D_f$): On held-out data, ELF computes rigorous $(1 - \delta)$ upper confidence bounds on the delayed-impact objective $g(\theta) = \tau - \mathbb{E}[I^{\pi_\theta} \mid c(X,Y,T)]$ using concentration inequalities (Student's $t$-test or Hoeffding’s inequality).
  5. Safe Fallback & Consistency: If the upper bound cannot statistically verify fairness ($U > 0$), the algorithm safely returns No Solution Found (NSF) rather than deploying an untrustworthy model. The authors prove that ELF bounds the probability of returning an unfair model to at most $\delta$ (Theorem 2) and is consistent—meaning the probability of returning a valid fair model approaches 1 as sample size $n \to \infty$ (Theorem 3).

Datasets

The authors evaluate ELF on real-world welfare data sourced from the National Data Archive on Child Abuse and Neglect (NDACAN, 2021):

  • AFCARS (Adoption and Foster Care Analysis and Reporting System): Youth demographic and foster-care case data.
  • NYTD (National Youth in Transition Database): Educational, financial, and employment outcomes as youth transition to adulthood.

To systematically test algorithms across varying degrees of predictability, the authors synthesize a controlled delayed-impact signal governed by parameter $\alpha \in [0, 1]$: $$I_i^\psi = \alpha \hat{Y}_i^\psi + (1 - \alpha)\mathcal{N}(\mu_T, \sigma_T^2)$$ Here, $\alpha$ regulates how strongly a decision affects delayed impact versus external demographic noise.

Key Insights and Ideas

  • Offline RL meets Fair Classification: Instead of simulating complex social systems, one can treat historical predictions and logged delayed metrics as bandit/RL trajectory data and apply off-policy evaluation.
  • Static Fairness Actively Fails Long-Term: Models constrained only by static criteria (demographic parity, equalized odds, predictive equality) consistently produce negative delayed impacts, demonstrating that static constraints are not merely suboptimal for long-term health—they are actively misleading.
  • Admitting Ignorance (NSF): Rather than gambling when data is scarce, providing an explicit NSF option allows models to maintain mathematical risk limits ($\le \delta$) regardless of sample size.

Key Results

  • Guaranteed Fairness (RQ1): Across 500 trials and across all values of $\alpha$, ELF maintained a 0% failure rate, safely under the allowed threshold ($\delta_0 = \delta_1 = 10\%$). In contrast, competing baselines (Fairlearn, Fairness Constraints, quasi-Seldonian static algorithms, and logistic regression) routinely failed the long-term fairness constraint on Black youth, showing failure rates between 60% and 100%.
  • Accuracy Trade-off (RQ2): With an explicit constraint requiring classification accuracy $\ge 75\%$, ELF reached 83% accuracy at $n = 1,000$ and scaled toward unconstrained baseline performance (~90%) as data increased. Baselines achieved $>90\%$ at smaller sample sizes only by deploying unfair models.
  • Data Efficiency: At $n = 4,096$, ELF successfully identified and returned a certifiably fair solution in 91% of runs.
  • Robustness Across Environmental Coupling (RQ3): Even when predictions had minimal influence on downstream outcomes (low $\alpha$), ELF never returned an unfair model; it simply required more data before passing the validation threshold.

Limitations and Future Work

  • Full Outcome Observability: ELF requires historical records containing delayed outcomes for both accepted and rejected decisions (or alternative treatment decisions). In settings like selective college admissions where non-admitted students disappear from administrative records, ELF cannot be applied directly without unobserved-counterfactual corrections.
  • Stationarity: Guarantees assume that the underlying societal data-generating distribution does not shift between historical logging and candidate deployment.
  • Multi-Step Dynamics: ELF currently handles single-step predictions with delayed outcomes. Extending off-policy long-term fairness guarantees to multi-stage Markov decision processes remains open for future research.

Summary updated

Abstract

Recent research has shown that seemingly fair machine learning models, when used to inform decisions that have an impact on people's lives or well-being (e.g., applications involving education, employment, and lending), can inadvertently increase social inequality in the long term. Existing fairness-aware algorithms consider static fairness constraints, such as equal opportunity or demographic parity, but enforcing constraints of this type may result in models that have a negative long-term impact on disadvantaged individuals and communities. We introduce ELF (Enforcing Long-term Fairness), the first classification algorithm that provides high-confidence fairness guarantees in terms of long-term, or delayed, impact. Importantly, ELF solves the open problem of providing such guarantees based only on historical data that includes observations of delayed impact. Prior methods, by contrast, require prior knowledge (or an estimate) of analytical models describing the relationship between a classifier's predictions and their corresponding delayed impact. We prove that ELF satisfies delayed-impact fairness constraints with high confidence and that it is guaranteed to identify a fair solution, if one exists, given sufficient data. We show empirically, using real-life data, that ELF can successfully mitigate long-term unfairness with high confidence.
BibTeX
@inproceedings{weber2023enforcing,
  author = {Aline Weber and Blossom Metevier and Yuriy Brun and Philip S. Thomas and Bruno Castro da Silva},
  title = {{Enforcing Delayed-Impact Fairness Guarantees}},
  booktitle = {International Conference on Learning Representations},
  year = {2023},
  url = {https://openreview.net/forum?id=tgAI50giBbg},
}
Figure