The Mathematical Incompatibility of Algorithmic Fairness Criteria

1,035 concepts505 papers1 proposal

When we design machine learning models to make high-stakes decisions—such as approving loans, hiring candidates, or diagnosing illnesses—we naturally want them to be "fair." However, translating the human concept of fairness into mathematical constraints reveals a frustrating reality: many intuitive definitions of fairness are mathematically incompatible with one another.

Unless a model is perfectly predictive or the underlying groups have identical characteristics, it is mathematically impossible to satisfy multiple common fairness criteria simultaneously. This realization has shifted the focus of the field from searching for a single "fair" algorithm to mapping out and navigating unavoidable ethical and statistical trade-offs.

The Core Conflict: Three Intuitive Metrics

To understand why fairness metrics clash, consider three of the most widely used statistical definitions of group fairness:

  1. Demographic Parity (Independence): The acceptance rate must be equal across all demographic groups. If $30\%$ of Group A is selected, $30\%$ of Group B must also be selected, regardless of the underlying distribution of true qualifications.
  2. Equalized Odds (Separation): The model's error rates must be equal across groups. Specifically, the True Positive Rate (TPR) and the False Positive Rate (FPR) must be identical for both Group A and Group B.
  3. Predictive Parity (Sufficiency / Calibration): A model's risk score must carry the same meaning for every group. If the model assigns a candidate a $70\%$ probability of success, that candidate should have a $70\%$ chance of actually succeeding, whether they belong to Group A or Group B.

As outlined in the comprehensive roadmap by Mengnan Du et al., these definitions represent fundamentally different ethical priorities. Crucially, they are mutually exclusive in most real-world scenarios.

The Root Cause: Differing Base Rates

The mathematical friction between these metrics is driven by differences in base rates—the baseline prevalence of the positive outcome across different groups in the historical data.

In FACT: A Diagnostic for Group Fairness Trade-offs, Joon Sik Kim et al. demonstrate this incompatibility using a unified mathematical framework called a confusion tensor. Imagine a scenario where Group A has a lower historical loan repayment rate (base rate) than Group B due to systemic socioeconomic disparities.

If we train a model that is perfectly calibrated (satisfying predictive parity), the model's predictions will faithfully reflect these base rates. Consequently, the model will assign lower risk scores on average to members of Group A, leading to a lower selection rate for Group A than Group B. This directly violates demographic parity.

If we attempt to force demographic parity by lowering the selection threshold for Group A to match Group B's acceptance rate, we introduce a new problem: we will inevitably accept individuals from Group A who have a lower probability of repayment than the individuals accepted from Group B. This immediately breaks calibration (a score of $70\%$ no longer means the same thing for both groups) and alters the false positive rates, violating equalized odds.

As proven by Omid Memarrast et al., whenever base rates differ between groups, you cannot simultaneously achieve demographic parity, equalized odds, and predictive parity. Practitioners are forced to choose which definition of equity to prioritize.

                  ┌────────────────────────┐
                  │ Differing Base Rates   │
                  │ (Historical Data Bias) │
                  └───────────┬────────────┘
                              │
              ┌───────────────┴───────────────┐
              ▼                               ▼
┌───────────────────────────┐   ┌───────────────────────────┐
│    Enforce Calibration    │   │ Enforce Demographic Parity│
│ (Predictive Parity holds) │   │ (Equal Selection Rates)   │
└─────────────┬─────────────┘   └─────────────┬─────────────┘
              │                               │
              ▼                               ▼
┌───────────────────────────┐   ┌───────────────────────────┐
│  Disparate Selection Rates│   │   Breaks Calibration &    │
│ (Demographic Parity fails)│   │   Equalized Odds Metrics  │
└───────────────────────────┘   └───────────────────────────┘

The Causal and Structural View

This incompatibility is not just an artifact of basic algebra; it is deeply rooted in the causal structure of the data. Babak Salimi et al. demonstrate that different fairness definitions require blocking contradictory paths within a causal graph.

For instance, ensuring that a decision is independent of a protected attribute (demographic parity) requires blocking all causal pathways from the sensitive attribute to the prediction. Conversely, ensuring that a prediction is conditionally independent given the true outcome (equalized odds) requires allowing certain pathways while blocking others. Because a single model cannot simultaneously block and permit the same causal dependencies, the resulting fairness constraints become mutually exclusive.

The Intersectional Escalation

The challenge intensifies when moving beyond a single binary attribute to multi-dimensional fairness or overlapping subgroups.

When trying to protect multiple sensitive attributes simultaneously (e.g., race, gender, and age), models experience a "seesaw" effect. Yi Sheng shows that optimizing a single model to be fair with respect to one sensitive attribute frequently causes a sharp decline in fairness for other attributes. This occurs because the optimization constraints for different demographic dimensions act as mutually antagonistic forces during training.

In deep learning architectures, this manifests as gradient interference. As documented by Wenlong Deng et al. in medical image classification, the gradients required to mitigate bias for one attribute can directly oppose and cancel out the gradients required for another, preventing the model from converging on a universally fair solution.

Furthermore, when considering intersectional subgroups (e.g., Black women), Guy N. Rothblum et al. prove that certain loss functions create mathematical paradoxes. Overlapping subgroups can impose entirely contradictory optimization requirements on the exact same individuals, making multi-group agnostic learning a profound theoretical challenge.

Because satisfying every fairness metric is mathematically impossible, the research community has shifted toward treating fairness as a multi-objective optimization problem. The goal is to find the Pareto frontier—the set of models where no single fairness or accuracy metric can be improved without degrading another.

To map this frontier, Alekh Agarwal et al. introduced a reductions approach. By transforming fair classification into a sequence of standard cost-sensitive classification problems, their framework allows practitioners to treat any machine learning algorithm as a black box and systematically explore the exact trade-offs between accuracy and diverse fairness constraints.

Accuracy
   ▲
   │      * (Unconstrained Model)
   │     . 
   │    .  <- Pareto Frontier (Best possible trade-offs)
   │   *
   │  .
   │ *
   │.
   └────────────────────────► Fairness Metric (e.g., Equalized Odds)

Similarly, Qingquan Zhang et al. leverage evolutionary multi-objective algorithms to generate a diverse ensemble of models along the frontier. Rather than forcing a single, arbitrary trade-off via hand-tuned loss weights, this approach presents decision-makers with a menu of optimized choices.

Ultimately, these mathematical boundaries mean that algorithmic fairness cannot be solved by computer science alone. As emphasized by Hamed Nilforoshan et al. and Stefan Feuerriegel et al., choosing where to sit on the Pareto frontier is a normative, value-based decision. It requires human stakeholders to weigh competing legal, social, and philosophical definitions of justice to determine which form of equity matters most in a given context.

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 521 papers
  2. Version 1 · 2026-10-04 · 358 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_fairness_metric_incompatibility_dfaccaa8,
  title = {fairness metric incompatibility},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/fairness-metric-incompatibility/rdr_dfaccaa86e5a49259035a81948b071fc},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 2, frozen text):

@misc{lacuna_fairness_metric_incompatibility_dfaccaa8_v2,
  title = {fairness metric incompatibility},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/fairness-metric-incompatibility/rdr_dfaccaa86e5a49259035a81948b071fc/v/2},
  note = {Version 2, frozen 2026-10-04. Accessed 2026-11-19},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake