Supposedly Fair Classification Systems and Their Impacts

Mackenzie Jorgensen, Elizabeth Black, Natalia Criado, Jose M. SuchAiOfAi @ IJCAI 2022 · OpenReview

Introduction

The pursuit of algorithmic fairness has become a cornerstone of modern machine learning research. However, the application of "fairness" constraints is not a guaranteed victory for social justice. This study demonstrates that common discrimination mitigation techniques often fail to improve—and can even worsen—the long-term financial well-being of disadvantaged groups. By focusing on "static" metrics at the moment of decision-making, practitioners may inadvertently cause "delayed impact" that harms the very populations they intend to protect.

The Discriminatory Impact Problem (DIP)

In algorithmic fairness, the standard approach is to ensure that a model's predictions are balanced across demographic groups at a single point in time. For instance, a bank might aim to hire men and women at equal rates. However, decisions have consequences that ripple through time.

The authors define the Discriminatory Impact Problem (DIP) as the divergence between a "fair" prediction today and its real-world outcome tomorrow. In the context of lending, a fairness constraint might compel a bank to grant more loans to a group statistically less likely to repay. While the initial offer appears fair, subsequent defaults can crash credit scores, leaving individuals in a worse financial position than if they had been denied the loan. This paper provides an empirical evaluation of how standard mitigation tools affect these long-term outcomes.

Methodology

To investigate this phenomenon, the researchers developed a comprehensive experimental pipeline to test the interactions between machine learning models, fairness metrics, and mitigation algorithms.

Experimental pipeline showing the integration of mitigation algorithms and fairness metrics into the model training process.

The pipeline includes:

  • ML Models: Standard classifiers such as Decision Trees, Gaussian Naive Bayes, Logistic Regression, and Gradient Boosted Trees.
  • Fairness Metrics: Five definitions including Demographic Parity and Error Rate Parity.
  • Mitigation Methods: "Reduction Algorithms" (Exponentiated Gradient and Grid Search) from the Fairlearn library, which optimize the trade-off between accuracy and fairness constraints.

To quantify Delayed Impact, the study employed a credit score update function: a successful repayment (True Positive) increases a score by 75 points, while a default (False Positive) results in a 150-point penalty.

Dataset Characteristics

The study utilized the FICO credit score dataset, a benchmark in fairness research. The data was transformed into a tabular format representing 100,000 individuals, with the Black group designated as "disadvantaged" (12% of the population) and the White group as "advantaged" (88%).

Distribution of repayment labels across the Black and White demographic groups, highlighting the initial disparity in the dataset.

As shown in the distribution, there is a significant disparity in the "Repay" vs. "Default" labels between the two groups. This baseline imbalance is what fairness mitigation techniques attempt to address, often with unintended consequences.

Results and Discussion

The experimental results revealed several critical insights into the nature of algorithmic fairness:

The Fairness Paradox

The most striking finding is that "fair" models frequently benefit the advantaged group while harming the disadvantaged one. In most scenarios, the White group's average credit scores improved slightly, while the Black group's scores plummeted compared to an unmitigated "unfair" model.

Metric Choice vs. Model Choice

The specific type of machine learning model used had negligible impact on the final outcome. Instead, the choice of fairness metric and mitigation algorithm were the primary determinants of long-term harm or benefit.

The Danger of Demographic Parity (DP)

Demographic Parity proved particularly damaging. By forcing equal acceptance rates regardless of qualification, it led to the "over-acceptance" of individuals unlikely to repay. This created a "self-fulfilling prophecy" where the financial health of the disadvantaged group was systematically eroded by the tool designed to help them. For example, using a Decision Tree with Exponentiated Gradient mitigation resulted in a staggering -712.78% change in impact for the Black group compared to the unmitigated model.

Conclusion

This research highlights that a "fair" prediction does not equate to a fair outcome. While Error Rate Parity (ERP) occasionally provided minimal positive impacts, most mitigation strategies resulted in significant long-term decline for disadvantaged groups. Future work must move beyond static metrics to consider the dynamic, long-term costs of algorithmic decisions, including the harms of both False Positives (predatory lending) and False Negatives (denied opportunity).

Summary updated

BibTeX
@inproceedings{jorgensen2022supposedly,
  author = {Mackenzie Jorgensen and Elizabeth Black and Natalia Criado and Jose M. Such},
  title = {{Supposedly Fair Classification Systems and Their Impacts}},
  booktitle = {AiOfAi @ IJCAI},
  year = {2022},
  url = {https://openreview.net/forum?id=kvkaxWEcHr},
}
Figure