Optimal Transport of Binary Classifiers to Fairness

Maarten Buyl, Tijl De Bie

Abstract

In past work on fairness in machine learning, the focus has been on forcing the prediction of classifiers to have similar statistical properties for people of different demographics. To reduce the violation of these properties, fairness methods usually simply rescale the classifier scores, ignoring similarities and dissimilarities between members of different groups. Yet, we hypothesize that such information is relevant in quantifying the unfairness of a given classifier. To validate this hypothesis, we introduce Optimal Transport to Fairness (OTF), a method that quantifies the violation of fairness constraints as the smallest Optimal Transport cost between a probabilistic classifier and any score function that satisfies these constraints. For a flexible class of linear fairness constraints, we construct a practical way to compute OTF as a differentiable fairness regularizer that can be added to any standard classification setting. Experiments show that OTF can be used to achieve an improved trade-off between predictive power and fairness.

Introduction

The development of ethical AI requires balancing predictive accuracy with fairness across demographic groups. Traditional group fairness methods often focus on equalizing statistical metrics, such as hiring rates, across different protected groups. However, these methods are frequently "feature-blind," ignoring the underlying similarities between individuals. This can lead to decisions that satisfy group-level parity but violate individual fairness—the principle that similar individuals should be treated similarly.

This paper introduces Optimal Transport to Fairness (OTF), a framework that bridges the gap between group and individual fairness. By leveraging Optimal Transport (OT) theory, OTF quantifies unfairness as the minimum "cost" required to transform a model's predictions into a state that satisfies specific fairness constraints. Crucially, this cost is informed by individual feature similarities, ensuring that the path to fairness respects the relationships between data points.

The OTF Framework

The core of the OTF approach is to treat the set of all fair scoring functions, denoted as $\mathcal{F}$, as a target space. The unfairness of a given classifier $h$ is defined as its distance to the nearest fair neighbor in $\mathcal{F}$.

To measure this distance, the authors employ Optimal Transport. OT calculates the minimum effort needed to transport mass from one distribution to another. In this context, it represents transporting the "score mass" of the current classifier to a fair version. A key innovation is the feature-aware cost function: the cost of moving a score from individual $i$ to individual $j$ is proportional to the difference in their non-protected features. Consequently, moving scores between similar individuals is "cheaper," encouraging the model to achieve group fairness by adjusting scores in a way that preserves individual-level consistency.

To make this framework practical for deep learning, the authors apply entropic smoothing to the OT problem, deriving a dual optimization formulation. This allows OTF to serve as a differentiable regularizer—a "fairness tax"—that can be integrated into standard loss functions and optimized via gradient descent.

Linear Fairness Notions

The OTF framework is designed for fairness notions that can be expressed as linear constraints. This includes common metrics like Demographic Parity (DP) and Equalized Odds (EO).

Table 1: Definitions of linear fairness constraints for Demographic Parity (DP) and Equalized Odds (EO).

As shown in Table 1, these constraints are formulated such that a model is considered fair when the expectation of certain functions $\mathbf{g}$ is zero. OTF then seeks the minimal transport to a distribution where these expectations are met.

Experimental Evaluation

The authors evaluated OTF on four benchmark datasets: UCI Adult (income prediction), UCI Bank Marketing (product subscription), Dutch Census (occupation prestige), and Diabetes (hospital readmission). These datasets include various sensitive attributes such as sex, race, and age.

Key Results

  • Superior Trade-offs: On the Adult and Diabetes datasets, OTF demonstrated a superior Pareto frontier compared to standard norm-based regularizers. It achieved higher AUC (Area Under the Curve) for the same level of fairness, particularly for Equalized Odds.
  • Handling Complexity: OTF successfully minimized unfairness for multiple attributes (Sex and Race) simultaneously on the Adult dataset, whereas baseline methods often struggled with the composition of different fairness notions.
  • The Power of Equalized Odds: The method is particularly effective for Equalized Odds. The authors suggest this is because OTF encourages the model to exchange scores between similar individuals who share the same ground-truth label.

Conclusion and Future Work

OTF represents a significant step forward by incorporating individual similarity into group fairness optimization. While computationally more intensive than feature-blind methods—with a complexity of $O(n(n + d_{\mathcal{F}}))$—it offers a more nuanced and effective way to enforce fairness. Future research may explore non-linear fairness constraints and the impact of different cost functions, such as learned distances, to further refine the accuracy-fairness trade-off.

Summary updated

Abstract

In past work on fairness in machine learning, the focus has been on forcing the prediction of classifiers to have similar statistical properties for people of different demographics. To reduce the violation of these properties, fairness methods usually simply rescale the classifier scores, ignoring similarities and dissimilarities between members of different groups. Yet, we hypothesize that such information is relevant in quantifying the unfairness of a given classifier. To validate this hypothesis, we introduce Optimal Transport to Fairness (OTF), a method that quantifies the violation of fairness constraints as the smallest Optimal Transport cost between a probabilistic classifier and any score function that satisfies these constraints. For a flexible class of linear fairness constraints, we construct a practical way to compute OTF as a differentiable fairness regularizer that can be added to any standard classification setting. Experiments show that OTF can be used to achieve an improved trade-off between predictive power and fairness.
BibTeX
@inproceedings{buyl2022optimal,
  author = {Maarten Buyl and Tijl De Bie},
  title = {{Optimal Transport of Binary Classifiers to Fairness}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2022},
  url = {https://openreview.net/forum?id=9VsBhTEuiO},
}
Figure