Aleatoric and Epistemic Discrimination in Classification

Hao Wang, Luxi He, Rui Gao, Flavio CalmonOpenReview 2023 · OpenReview

Abstract

Machine learning (ML) models can underperform on certain population groups due to choices made during model development and bias inherent in the data. We categorize sources of discrimination in the ML pipeline into two classes: aleatoric discrimination, which is inherent in the data distribution, and epistemic discrimination, which is due to decisions during model development. We quantify aleatoric discrimination by determining the performance limits of a model under fairness constraints, assuming perfect knowledge of the data distribution. We demonstrate how to characterize aleatoric discrimination by applying Blackwell's results on comparing statistical experiments. We then quantify epistemic discrimination as the gap between a model's accuracy given fairness constraints and the limit posed by aleatoric discrimination. We apply this approach to benchmark existing interventions and investigate fairness risks in data with missing values. Our results indicate that state-of-the-art fairness interventions are effective at removing epistemic discrimination. However, when data has missing values, there is still significant room for improvement in handling aleatoric discrimination.

Introduction

The field of Fair Machine Learning has produced hundreds of algorithms designed to reduce bias. However, researchers have lacked a fundamental way to determine if these algorithms are truly optimal. When a model performs poorly on a minority group, is it due to suboptimal model choices (epistemic discrimination) or because the underlying data lacks the information necessary for fair, accurate predictions (aleatoric discrimination)?

This paper introduces a framework to disentangle these sources of bias by establishing the Fairness Pareto Frontier (FairFront)—the theoretical maximum accuracy possible for any given fairness constraint. This provides a "speed limit" for fairness interventions, helping practitioners decide whether to focus on algorithmic refinement or improved data collection.

Theoretical Framework and Methods

The authors define the FairFront as the upper bound of achievable accuracy under group fairness constraints. Common metrics addressed include Statistical Parity (SP), Equalized Odds (EO), and Overall Accuracy Equality (OAE).

Definitions of group fairness metrics: Statistical Parity (SP), Equalized Odds (EO), and Overall Accuracy Equality (OAE).

To calculate this frontier without exhaustive testing, the framework utilizes Blackwell’s Comparison of Experiments (1951, 1953) to characterize the feasible region of confusion matrices for a given data distribution. The search for the FairFront is formulated as a convex optimization problem. Since this problem can involve infinitely many constraints, the authors introduce a Greedy Improvement Algorithm that uses "Difference of Convex" (DC) programming to iteratively tighten the upper bound until convergence.

Experimental Evaluation

The approach was evaluated on standard datasets including UCI Adult, ProPublica COMPAS, German Credit, and the High School Longitudinal Study (HSLS). A primary finding is that state-of-the-art (SOTA) interventions, such as Reduction and FairProjection, are remarkably close to the theoretical FairFront on standard datasets.

Accuracy vs. Max Equalized Odds for Adult and COMPAS datasets, comparing the theoretical FairFront with baseline and SOTA interventions.

As illustrated above, the performance of SOTA methods nearly matches the theoretical upper bound on the Adult and COMPAS datasets. This suggests that further algorithmic "tweaking" on these specific benchmarks may yield negligible gains, as the remaining discrimination is primarily aleatoric.

Key Insights: Data vs. Algorithms

The research highlights a critical "data is the problem" moment. When data has missing values—particularly when missingness rates differ across groups—aleatoric discrimination increases significantly. For instance, in experiments on the Adult dataset, increasing the missing data probability for one group from 10% to 70% caused a sharp decline in the FairFront. This proves that no algorithm can "fix" fundamental data quality issues.

By quantifying the gap between a model's performance and the FairFront, practitioners can diagnose their next move. A large gap indicates a need for better models (reducing epistemic bias), while a small gap despite poor performance indicates a need for better data collection (reducing aleatoric bias).

Limitations and Future Work

While effective for standard benchmarks, the method faces challenges with high-dimensional categorical data due to the exponential growth of the distribution's support size. Additionally, the current algorithm assumes perfect knowledge of the data distribution; in practice, estimation errors from finite datasets can affect the tightness of the bound. Future research will explore extending this framework to individual fairness and optimizing the piecewise linear approximations used in the frontier calculation.

Summary updated

Abstract

Machine learning (ML) models can underperform on certain population groups due to choices made during model development and bias inherent in the data. We categorize sources of discrimination in the ML pipeline into two classes: aleatoric discrimination, which is inherent in the data distribution, and epistemic discrimination, which is due to decisions during model development. We quantify aleatoric discrimination by determining the performance limits of a model under fairness constraints, assuming perfect knowledge of the data distribution. We demonstrate how to characterize aleatoric discrimination by applying Blackwell’s results on comparing statistical experiments. We then quantify epistemic discrimination as the gap between a model’s accuracy given fairness constraints and the limit posed by aleatoric discrimination. We apply this approach to benchmark existing interventions and investigate fairness risks in data with missing values. Our results indicate that state-of-the-art fairness interventions are effective at removing epistemic discrimination. However, when data has missing values, there is still significant room for improvement in handling aleatoric discrimination.
BibTeX
@misc{wang2023aleatoric,
  author = {Hao Wang and Luxi He and Rui Gao and Flavio Calmon},
  title = {{Aleatoric and Epistemic Discrimination in Classification}},
  year = {2023},
  url = {https://openreview.net/forum?id=u52SKyPUF7B},
  howpublished = {OpenReview},
}
Figure