The Delayed Societal Cost of Unfairness

Giovani Valdrighi, Isabel Valera, Marcos Medeiros RaimundoarXiv 2026 · arXiv

In the world of fair machine learning, we are often taught that there is no free lunch. If you want a model to be fairer to a disadvantaged group, you must be prepared to sacrifice some accuracy or profit. This is the "performance-fairness trade-off," and it’s usually visualized as a Pareto frontier—a curve where you can’t improve one objective without hurting the other.

However, a new paper titled "The Delayed Societal Cost of Unfairness" argues that this trade-off is often an illusion caused by looking at static data. By accounting for how decisions change the world over time, the authors show that being "too fair" on paper might actually be the most profitable strategy in the long run.

TL;DR

Standard fairness metrics ignore how decisions (like granting a loan) reshape the population (e.g., by improving credit scores). This paper proves that when these dynamics are considered, fairer policies can "dominate" less fair ones—improving outcomes for both the decision-maker and the subjects simultaneously.

What Problem Does This Paper Address?

Most fairness research evaluates models on a "snapshot" of data. But in high-stakes domains like lending, hiring, or college admissions, the model's decisions are not passive; they are interventions.

Imagine a bank using a credit score threshold. If they lower the threshold for a disadvantaged group (a fairness intervention), those individuals get loans. If they repay, their credit scores go up, making them more "qualified" for future interactions. Conversely, if they are rejected, their scores may stagnate or drop.

The authors tackle a critical question: When does a policy that looks optimal on static data become suboptimal once it starts shaping the population? They specifically look for cases of "post-deployment dominance," where a fairer policy ends up being better for everyone than the "optimal" unfair one.

Methods

The researchers use the framework of Performative Prediction, where a model's deployment changes the underlying data distribution. They model a binary decision setting (e.g., Loan vs. No Loan) with two demographic groups.

Key components of their approach include:

  • Selection Dynamics: They adapt a model where a subject's "qualification score" updates based on the decision. Acceptance + Success = Score Increase; Acceptance + Failure = Score Decrease; Rejection = Score stays the same (or changes slightly).
  • Multi-Objective Optimization: They treat Decision-Maker (DM) loss and Unfairness as two competing objectives, controlled by a fairness weight ($\lambda$).
  • Fairness Notions: They test both Rawlsian fairness (which focuses on helping the worst-off group) and Ealitarian fairness (which seeks to equalize costs across groups).

Datasets

The authors validate their theoretical findings using the FICO credit score dataset. This dataset contains real-world credit scores and repayment rates for Black and white consumers in the US. It is a standard benchmark for studying long-term fairness because it clearly shows the initial disparity in score distributions between groups.

Key Insights and Ideas

  • The Static Trap: A policy that is Pareto-optimal on pre-deployment data can be "dominated" after deployment. This means there exists a fairer policy that, once the population adapts, results in lower loss for the bank and better outcomes for the applicants.
  • The Break-Even Benefit: If accepting a disadvantaged person at the "break-even" point (where expected profit is zero) leads to them becoming more qualified on average, then the bank is actually losing money in the long run by not being fairer.
  • Perverse Incentives of Accuracy: One of the most surprising findings is that if a decision-maker only cares about accuracy (correctly predicting success/failure), they might actually benefit from a population becoming less qualified. Why? Because it's very easy to "accurately" reject someone who is clearly unqualified. This highlights why accuracy alone is a dangerous metric for societal impact.
This figure illustrates how the Pareto front of "optimal" policies on static data (left) can collapse after deployment (right). The pink dots represent policies that were once considered optimal but are now "dominated" by fairer, more efficient alternatives.

Key Results

  • Dominance is Real: On the FICO data, the authors found that small fairness weights are almost always dominated. There is a minimal fairness weight ($\lambda_{min}$) below which a policy is strictly worse than a fairer alternative.
  • The Power of Repayment: The range of "dominated" policies grows as the benefit of successful repayment ($b^+$) increases relative to the cost of default ($b^-$). If a loan helps someone a lot, the bank should be much more willing to take a "fairness risk."
  • Rawlsian vs. Egalitarian: Rawlsian fairness is more likely to lead to "win-win" scenarios because it never requires the decision-maker to intentionally "worsen" the outcomes for the advantaged group just to achieve balance.
Figure 3: Changes (indicated by arrows) of a DS from the disadvantaged group (marked by the black dot •) that is rejected by the pre-deployment DM optimal policy θ(0) but accepted by another policy θ(λ) with λ > 0. Their score updates to a new value based on acceptance/rejection and repayment/default. Left: increasing λ moves the threshold from θ(0) to θ(λ), accepting the DS in the shaded band. Center: the DM’s cost of the best decision for any DS (x, g), which is independent of the marginal distribution of scores. A DS in the band who repays moves to a higher score with less risk for the DM; ∆Γ is the DM’s expected gain from accepting rather than rejecting. Right: the DS cost, which is non-null with rejection. A DS that repays increases their score and remains accepted, while one that defaults drops below the threshold θ(λ) and gets rejected, as it already was under θ(0).

Limitations and Future Work

The authors acknowledge several limitations:

  1. One-Step Dynamics: The model looks at a single "before and after" step. Real-world feedback loops are continuous and multi-round.
  2. Stylized Updates: The linear score update ($X + b^+$) is a simplification of complex social and economic mobility.
  3. Knowledge of Dynamics: The theory assumes the decision-maker knows exactly how the population will react. In practice, these dynamics are often unknown and must be learned.

Future work involves extending this to multi-round settings and developing algorithms that can learn these population dynamics without exposing subjects to harmful "experimental" decisions.

Summary updated

Abstract

Fair machine learning typically evaluates the societal costs of a decision policy -- the loss incurred by the decision-maker (DM) and the unfair distribution of costs among demographic groups of decision-subjects (DS) -- on static data, where reducing one often comes at the expense of the other. Yet decisions change the populations on which they act, and prior work has shown that fairness interventions designed on static data may fail to benefit, or even harm, the disadvantaged group over time. We investigate a complementary question: When can a policy that is Pareto-optimal on pre-deployment data be outperformed by a fairer policy once each is evaluated on the population it induces, reducing costs for one stakeholder without increasing them for the other? In a setting where qualification scores evolve in response to binary decisions and their outcomes, we show that under Rawlsian fairness, which never lowers any group's acceptance rate, placing sufficiently high weight on fairness reduces societal costs whenever accepting disadvantaged DS at the DM's break-even point leaves them in a state less costly to the DM than rejecting them. Under egalitarian fairness, which may lower the advantaged group's acceptance rate, this additionally requires the reduction in DM loss on the disadvantaged group to outweigh the increase on the advantaged group. Under both fairness notions, however, when DM loss is accuracy-based, improvements in the post-deployment population may arise from declining DS qualification, revealing that gains for the DM do not necessarily translate into better outcomes for DS. Our results show that policy-induced population dynamics can overturn apparent performance-fairness trade-offs, allowing fairer policies to reduce the delayed societal costs of decision-making.
BibTeX
@misc{valdrighi2026delayed,
  author = {Giovani Valdrighi and Isabel Valera and Marcos Medeiros Raimundo},
  title = {{The Delayed Societal Cost of Unfairness}},
  year = {2026},
  eprint = {2611.26030},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.26030},
}
Figure