TL;DR
This paper proposes a novel framework for quantifying algorithmic unfairness by adapting inequality indices from economics. It allows for a unified measurement of both individual and group unfairness, revealing critical trade-offs between different fairness notions and accuracy that were previously overlooked.
What Problem Does This Paper Address?
Algorithmic decision-making systems are increasingly used in high-stakes domains like criminal justice and credit scoring, raising concerns about fairness. While many studies define conditions for fairness, they often lack a principled way to measure the degree of unfairness. This makes it difficult to answer crucial questions like: "Which of two unfair algorithms is more unfair?" or "How do different fairness interventions impact overall unfairness?" Existing approaches often focus solely on group-level fairness (e.g., equalizing false positive rates across demographic groups) and may not adequately capture individual-level unfairness or the complex interplay between different fairness dimensions.
Methods
The core innovation of this paper is the application of inequality indices from economics (like the Gini coefficient or Theil index, specifically using the generalized entropy family) to measure algorithmic unfairness. The authors propose:
- Defining a "benefit function": This function maps an algorithm's outcome for an individual to a numerical benefit, tailored to the specific fairness notion being considered (e.g., a positive outcome might be more beneficial than a negative one, or an accurate prediction might be more beneficial than an inaccurate one).
- Quantifying unfairness with inequality indices: By plugging these benefit values into established inequality indices, the authors can quantify how unequally benefits are distributed across individuals or groups.
- Decomposition of unfairness: A key feature of the chosen inequality indices is their subgroup decomposability. This allows the overall individual-level unfairness to be broken down into two components:
- Between-group unfairness: Measures inequality between the average benefits received by different predefined groups.
- Within-group unfairness: Measures inequality among individuals within each group.
This decomposition is crucial for understanding trade-offs.
Datasets
The authors empirically validate their approach using two real-world datasets:
- Adult income dataset: Used for predicting whether an individual earns over $50,000 per year. Sensitive features considered include gender and race.
- ProPublica COMPAS dataset: Used for predicting criminal recidivism. Sensitive features considered include gender, race, and age.
Key Insights and Ideas
- A unified measure of unfairness: By leveraging economic inequality indices, the paper provides a single, principled framework to quantify unfairness at both individual and group levels, allowing for direct comparison of algorithms.
- The danger of focusing solely on group fairness: Minimizing only between-group unfairness (a common practice) can paradoxically increase within-group and, consequently, overall individual unfairness. This highlights a critical blind spot in many existing fairness interventions.
- Granularity matters for group fairness: The contribution of between-group unfairness to overall unfairness increases with the number of defined groups (e.g., considering gender and race leads to a higher between-group component than considering just gender). This suggests that "fairness gerrymandering" can occur if groups are not carefully considered.
- Trade-offs are inherent: The paper demonstrates that optimizing for accuracy and fairness are not always aligned, and that different components of unfairness (between-group vs. within-group) can be in tension.
Key Results
- Theoretical characterization of trade-offs: The paper provides theoretical propositions showing that accuracy-optimal and fairness-optimal classifiers may not coincide, and that minimizing between-group unfairness can lead to increased within-group unfairness.
- Empirical validation: Experiments on the Adult and COMPAS datasets confirm these theoretical findings. For instance, Figure 5 shows that between-group unfairness is often a very small fraction of overall unfairness, meaning interventions targeting only the former might have limited impact on the latter. Figure 7 demonstrates that applying constraints to reduce between-group unfairness can indeed increase within-group and overall unfairness.
- Illustrative examples: The paper provides concrete examples showing how adding features can sometimes worsen unfairness, and how the fairness of an algorithm can be arbitrarily worse than the accuracy-optimal one.
Limitations and Future Work
- Choice of benefit function: The paper acknowledges that the choice of the benefit function is context-dependent and crucial for defining what constitutes "benefit" for different fairness notions.
- Generalizability beyond binary classification: While the framework is presented for binary classification, the authors note it can be extended to multiclass and regression problems, as well as unsupervised tasks, though this requires careful specification of benefit functions.
- Computational complexity: The paper hints that as the number of groups increases, controlling between-group unfairness becomes computationally harder.
- Future work: The authors suggest exploring the application of their framework to unsupervised learning tasks and further investigating the impact of different benefit function choices. They also point to the need for more nuanced fairness interventions that consider the interplay between different unfairness components.
Summary updated