Abstract
Most works on the fairness of machine learning systems focus on the blind optimization of common fairness metrics, such as Demographic Parity and Equalized Odds. In this paper, we conduct a comparative study of several bias mitigation approaches to investigate their behaviors at a fine grain, the prediction level. Our objective is to characterize the differences between fair models obtained with different approaches. With comparable performances in fairness and accuracy, are the different bias mitigation approaches impacting a similar number of individuals? Do they mitigate bias in a similar way? Do they affect the same individuals when debiasing a model? Our findings show that bias mitigation approaches differ a lot in their strategies, both in the number of impacted individuals and the populations targeted. More surprisingly, we show these results even apply for several runs of the same mitigation approach. These findings raise questions about the limitations of the current group fairness metrics, as well as the arbitrariness, hence unfairness, of the whole debiasing process.
Introduction
In the field of algorithmic fairness, practitioners typically aim for global metrics like Demographic Parity (equal acceptance rates) or Equalized Odds (equal error rates). However, these aggregate scores are often "underspecified." Multiple models can achieve identical fairness and accuracy scores on a corporate dashboard while making drastically different decisions for the same individuals. This phenomenon, known as predictive multiplicity, means that the choice of which "fair" model to deploy is often arbitrary, potentially introducing new forms of unfairness at the individual level.
The FRAME Framework
To pull back the curtain on the debiasing process, the authors introduce FRAME (FaiRness Arbitrariness and Multiplicity Evaluation). Instead of relying on a single global score, FRAME audits the transition from a biased baseline to a fair model across five dimensions:
- Impact Size: The percentage of the population whose predictions were changed to achieve fairness.
- Change Direction: Whether fairness is achieved by "leveling up" (accepting more disadvantaged individuals) or "leveling down" (rejecting more privileged individuals).
- Decision Rates: How the overall acceptance rate of the model shifts.
- Affected Subpopulations: Identifying specific types of people whose outcomes were altered using explainability tools.
- Neglected Subpopulations: Detecting "local" unfairness that persists within specific data slices despite global parity.
Key Findings
The Disagreement Problem
The study found that models with nearly identical aggregate scores often have a surprisingly low Intersection over Union (IOU). In some cases, the overlap between individuals targeted by different "fair" models was less than 9%. This suggests that different algorithms are "fixing" bias by targeting completely different people, making the outcome for any single individual highly dependent on the choice of algorithm.
Impact Size and Stability
The "cost" of fairness in terms of individual prediction changes varies wildly. As shown in the figure below, some methods like EGR require changing up to 31% of predictions on certain datasets, while others are far more efficient. Furthermore, non-deterministic methods (like Adversarial Debiasing) exhibit significant instability; in the Credit dataset, only 2% of the individuals targeted for a change were consistently affected across multiple training runs.
Global vs. Local Fairness
A critical insight is that global fairness does not guarantee local fairness. A model can achieve perfect aggregate scores while remaining highly unfair to specific subgroups. For example, in the Adult dataset, most debiasing methods failed to improve fairness for individuals under 26, despite looking "fair" on the dataset as a whole.
Conclusion
The findings reveal that bias mitigation is not a neutral technical process. The arbitrariness of which individuals are affected by "fairness" interventions raises significant ethical and legal questions. Future research should prioritize "change-efficient" debiasing algorithms and develop secondary criteria to help practitioners choose between multiple fair models in a non-arbitrary, socially responsible way.
Summary updated