How Fair AI Can Make Us Richer

Sandra WachterEuropean 2021 · DOI

Abstract

Artificial intelligence (AI) has become an integral part of modern life, moving far beyond mundane tasks like search results or music suggestions. Today, algorithms steer critical life outcomes: university admissions, hiring decisions, insurance premiums, social benefits, and even judicial sentencing. However, as these systems scale, they often inherit and amplify societal biases. This paper argues that AI bias is not merely a social justice concern but a significant economic inefficiency. By shifting from "bias-preserving" to "bias-transforming" metrics, we can move beyond the status quo to unlock untapped human potential and create a wealthier, more equitable society.

The Blind Spot of Algorithmic Discrimination

Current non-discrimination laws are ill-equipped to handle the nuances of machine learning. Traditional legal frameworks rely on a complaint-based system: an individual feels wronged, observes a peer receiving better treatment, and seeks legal recourse.

AI disrupts this model by operating "behind our backs." When an algorithm denies a loan or suppresses a job advertisement based on browsing history or inferred traits, the victim rarely knows the event even occurred. Furthermore, AI frequently utilizes "untraditional" data—such as shopping habits or social media activity—that lacks an intuitive link to protected attributes like race or gender. This opacity makes it nearly impossible for human observers to detect discriminatory patterns without specialized technical intervention, creating a "blind spot" where systemic inequality can flourish unchecked.

A Conceptual Framework for Fairness

To address these challenges, we must categorize how we measure fairness. Existing technical bias tests generally fall into two categories:

  1. Bias-Preserving Tests: These metrics evaluate fairness based on equal error rates across different groups. They essentially treat historical data as a neutral and accurate representation of merit. If the data shows a specific group has historically underperformed, the algorithm accepts this as "ground truth."
  2. Bias-Transforming Tests: These focus on parity of outcomes. They do not take the status quo for granted; instead, they flag disparities regardless of whether the underlying historical data "justifies" them.

The paper advocates for Conditional Demographic Disparity (CDD) as a superior metric, particularly within the European legal context. CDD aligns with the goal of "substantive equality"—the principle that the law should actively dismantle existing inequalities rather than merely ignoring them.

The Economic Case: Fairness as an Investment

A common misconception is the "Equality as Charity" myth—the idea that fairness is a "tax" on innovation or a handout to the unqualified. This research flips that narrative: fairness is a strategic investment.

When an algorithm rejects a qualified candidate because they do not match the profile of previous successes, the organization loses talent and money. We often treat metrics like grades, reference letters, and salary histories as objective truths, but these are actually "broken proxies" contaminated by historical bias. For example, gendered language in reference letters often describes women as "hard-working" while men are labeled "geniuses." By using bias-transforming tests, companies can identify "hidden gems"—talented individuals who currently fall through the cracks of biased traditional metrics.

AI as a Societal Mirror

Rather than viewing AI as a fixed, objective tool, we should see it as a mirror reflecting our societal failings. Algorithmic audits allow us to identify where human selection strategies—in hiring, lending, and beyond—have been failing for decades.

However, there is no "one size fits all" mathematical definition of fairness. The appropriate test depends heavily on the sector, the jurisdiction, and the specific goals of the system. Implementing these changes requires moving beyond simple code fixes to a broader re-evaluation of how we define "merit" and "talent."

Conclusion and Future Directions

Transitioning to fairer AI systems offers a qualitative roadmap for both legal compliance and economic growth. While "Bias-Transforming" tests satisfy the legal mandate for substantive equality, they also provide a clear financial incentive for the private sector by optimizing human capital.

The primary hurdle remains the "buy-in" problem; ethical arguments often fail to change corporate behavior unless they are linked to profitability. Future work must focus on the implementation complexity of metrics like CDD, specifically in choosing the right "conditioning variables" to ensure that fairness interventions are both legally sound and economically productive. By refining these tools, we can use AI not just to automate the past, but to build a more prosperous future.

Summary updated

Abstract

We are all aware that artificial intelligence (AI) has now become an integral part of our lives. AI systems are behind mundane tasks such as displaying search results on Safari, preparing travel routs on Google and suggesting new music on Spotify. But algorithms also steer important parts of our lives: if we get admitted to university, if we are hired, fired or promoted, if we get insurance, social benefits or a loan, or even if we have to go to prison. Algorithms can touch almost every aspect of our lives.
BibTeX
@inproceedings{wachter2021how,
  author = {Sandra Wachter},
  title = {{How Fair AI Can Make Us Richer}},
  booktitle = {European},
  year = {2021},
  doi = {10.21552/edpl/2021/3/5},
}
Figure