Optimal Transport of Classifiers to Fairness

Maarten Buyl, Tijl De Bie

Introduction

In the pursuit of equitable machine learning, researchers often focus on forcing a model's predictions to exhibit similar statistical properties across different demographic groups. However, traditional methods for reducing fairness violations typically rely on simply rescaling classifier scores, ignoring the underlying similarities and dissimilarities between the individuals themselves.

To address this gap, this paper introduces Optimal Transport to Fairness (OTF), a novel, differentiable regularizer. OTF penalizes unfairness by measuring the Optimal Transport cost between a model's current predictions and the nearest perfectly fair score distribution. By factoring in the similarity of input features, OTF achieves a superior trade-off between predictive accuracy and fairness, particularly for complex constraints like Equalized Odds.

The Problem: Ignoring Input Features

Machine learning models often inadvertently discriminate against protected groups. A common mitigation strategy is "inprocessing," where a fairness regularizer is added to the model's loss function during training to penalize differences in average prediction scores between demographics (e.g., enforcing Demographic Parity).

However, traditional regularizers only evaluate the statistical properties of the model's outputs. They completely ignore the input features that generated those scores. The authors argue this is a critical missed opportunity: unfairness is arguably more egregious when two individuals possess very similar features but receive vastly different scores simply due to their protected status. Existing methods treat all score adjustments equally, failing to capture the nuance of individual similarity.

Methodology: Optimal Transport to Fairness

To make fairness feature-aware, the authors formulate unfairness as an Optimal Transport (OT) problem. They measure the minimum "work" required to transport a classifier's current probability scores to a hypothetical set of scores that perfectly satisfies a given fairness constraint.

Linear fairness constraints for Demographic Parity and Equalized Odds.

The core technical contributions of this approach include:

  • The OTF Metric: By defining the OT cost function as the distance between individuals' non-protected features, the model is penalized more heavily if it must transfer scores between highly dissimilar people to achieve fairness.
  • Entropic Smoothing: Because standard OT is not differentiable (and thus unusable for gradient descent), the authors apply entropic regularization to make the cost differentiable ($\text{OTF}_\epsilon$).
  • The Adjustment Term: Smoothed OT has a known quirk: the cost doesn't equal zero even if the two distributions are identical, due to the entropy term. The authors introduce an adjustment term ($\text{OTFR}_\epsilon$) that, when subtracted, yields a tight lower bound ($\text{OTF}_\epsilon^0$) that correctly zeroes out when the model is already fair.
  • Dual Optimization: This adjusted cost is added to the standard cross-entropy loss and optimized efficiently using a dual formulation and exact coordinate ascent on data batches.

Key Insights and Advantages

Bridging Group and Individual Fairness

By using feature distance to guide the transport of scores, OTF naturally incentivizes the model to achieve group fairness by adjusting the scores of similar individuals across different groups. This elegantly bridges the gap between group fairness (equal outcomes across demographics) and individual fairness (treating similar people similarly).

Synergy with Equalized Odds

OTF is particularly effective when optimizing for Probabilistic Equalized Odds (PEO). Because PEO conditions fairness on the true target label, the individuals being compared naturally have more similar features. This aligns perfectly with OTF's feature-aware transport cost, rewarding the model for making sensible, localized score adjustments.

Unmatched Flexibility

Because OTF relies on a flexible matrix of linear constraints, it can easily handle multiple sensitive attributes simultaneously (e.g., SEX and RACE together), as well as continuous sensitive attributes (like exact AGE). Many existing OT-based fairness methods mathematically break down when faced with continuous or overlapping protected groups.

Experimental Results

The authors evaluated OTF on four standard tabular datasets: UCI Adult, UCI Bank Marketing, Dutch Census, and Diabetes.

  • Demographic Parity: When optimizing for Probabilistic Demographic Parity (PDP), OTF matched the performance (AUC vs. fairness trade-off) of standard baseline regularizers.
  • Equalized Odds: When optimizing for PEO, OTF significantly outperformed the standard $L^1$ norm baseline on the Adult and Diabetes datasets, maintaining higher predictive accuracy for the same level of fairness violation.
  • Complex Attributes: OTF successfully optimized fairness for continuous AGE on the Bank dataset and simultaneously for SEX and RACE on the Adult dataset—configurations where baseline OT methods (like Barycenter approaches) could not even be run due to architectural limitations.

Limitations and Future Work

While powerful, OTF is more computationally expensive than simple norm-based regularizers. It scales at $O(n(n+d_\mathcal{F}))$ where $n$ is the batch size and $d_\mathcal{F}$ is the number of fairness constraints, compared to $O(nd_\mathcal{F})$ for simpler methods. Additionally, the current mathematical formulation only supports linear fairness notions and cannot handle non-linear concepts like predictive sufficiency.

Future work may extend this framework to non-linear fairness notions and explore creative OT cost functions that explicitly direct the transport of classifier scores to further enhance individual fairness.

Summary updated

Abstract

In past work on fairness in machine learning, the focus has been on forcing the prediction of classifiers to have similar statistical properties for people of different demographics. To reduce the violation of these properties, fairness methods usually simply rescale the classifier scores, ignoring similarities and dissimilarities between members of different groups. Yet, we hypothesize that such information is relevant in quantifying the unfairness of a given classifier. To validate this hypothesis, we introduce Optimal Transport to Fairness (OTF), a method that quantifies the violation of fairness constraints as the smallest Optimal Transport cost between a probabilistic classifier and any score function that satisfies these constraints. For a flexible class of linear fairness constraints, we construct a practical way to compute OTF as a differentiable fairness regularizer that can be added to any standard classification setting. Experiments show that OTF can be used to achieve an improved trade-off between predictive power and fairness.
BibTeX
@inproceedings{buyl2022optimal,
  author = {Maarten Buyl and Tijl De Bie},
  title = {{Optimal Transport of Classifiers to Fairness}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2022},
  url = {https://openreview.net/forum?id=9VsBhTEuiO},
}
Figure