Traditional algorithmic fairness often relies on matching summary statistics across demographic groups—for instance, ensuring that the average prediction or selection rate is equal for everyone. However, as Silvia Chiappa points out, aligning only the average outcomes fails to account for disparities in the full shape or variance of the outcome distributions. If two groups have the same mean score but completely different distributions, a classifier thresholded at a specific point will still treat them drastically unequally. Furthermore, models optimized for a single threshold are notoriously brittle; shifting the decision cutoff can instantly destroy any fairness guarantees.
To build truly robust equity, a growing body of research reframes fairness as a distributional alignment problem. Instead of matching isolated metrics, the goal is to make the entire probability distribution of model predictions, latent representations, or features statistically indistinguishable across different sensitive groups.
The Geometry of Fairness via Optimal Transport
How do we mathematically force two or more distinct distributions to look identical without destroying the predictive utility of the model? The most geometrically intuitive tool for this task is Optimal Transport (OT).
Instead of treating probability distributions as abstract functions, OT views them as physical piles of mass. The distance between two distributions—often measured via the Wasserstein metric—is defined as the minimum "work" required to morph one pile into the other. By incorporating a Wasserstein distance penalty directly into a neural network's loss function, as demonstrated by Laurent Risser et al. in Tackling Algorithmic Bias in Neural-Network Classifiers using Wasserstein-2 Regularization, the model is forced to find a representation space where the cost of transforming one group's distribution into another is minimized.
When dealing with multiple demographic groups, aligning them pairwise can become contradictory or computationally intractable. To solve this, researchers utilize the Wasserstein barycenter (or Fréchet mean)—the mathematically optimal "compromise" distribution that minimizes the total transport distance from all individual group distributions.
[Group A Distribution] \
--> [Wasserstein Barycenter] <-- [Shared Fair Target]
[Group B Distribution] /
As proposed by Ruicheng Xian et al. in Differentially Private Post-Processing for Fair Regression and Aldo Pacchiano et al. in Wasserstein Fair Classification, the barycenter acts as a universal, fair reference target. Biased group-specific predictions are mapped toward this shared barycenter via optimal transport maps. Because the barycenter is computed globally, it preserves as much of the original predictive information as possible while ensuring perfect statistical parity.
This geometric perspective also provides an elegant way to navigate the inevitable trade-off between model accuracy and fairness. By tracing a Wasserstein geodesic—a straight path in the space of probability measures—between the original biased distribution and the fair barycenter, Shizhou Xu et al. in Fair Data Representation for Machine Learning at the Pareto Frontier show that practitioners can use McCann interpolation to smoothly select any desired point along the Pareto frontier.
Capturing Higher-Order Disparities with Kernels and Divergences
While Optimal Transport offers powerful geometric guarantees, computing exact Wasserstein distances over high-dimensional neural network embeddings can be computationally expensive or suffer from vanishing gradients when distributions do not overlap. To circumvent this, researchers frequently employ alternative statistical distances that act as differentiable proxies for alignment.
One prominent approach is Maximum Mean Discrepancy (MMD). By mapping data distributions into a high-dimensional Reproducing Kernel Hilbert Space (RKHS) using universal kernels, MMD can detect and eliminate subtle, non-linear differences between groups that go beyond simple mean and covariance matching. Yijin Ni et al. in Kernel-based Equalized Odds: A Quantification of Accuracy-Fairness Trade-off in Fair Representation Learning demonstrate that universal kernels allow MMD to capture complex disparities that standard linear measures miss, providing a highly stable objective for training fair neural networks. Similarly, Junghyun Lee et al. in Fast and Efficient MMD-Based Fair PCA via Optimization over Stiefel Manifold leverage MMD to ensure that low-dimensional representations do not implicitly preserve protected attributes through higher-order statistical moments.
Alternatively, information-theoretic measures like $f$-divergences and Kullback-Leibler (KL) divergence can be leveraged to enforce independence. For example, Mislav Balunovic et al. in Fair Normalizing Flows use symmetrized KL divergence to minimize the discrepancy between latent distributions, mathematically bounding the maximum accuracy of any potential adversary attempting to recover sensitive attributes. As argued by Meiyu Zhong et al. in Learning Fair Classifiers via Min-Max F-divergence Regularization, minimizing these non-linear divergences provides a much stronger guarantee of statistical independence than traditional linear correlation constraints.
Implementation Across the Pipeline
The beauty of the distributional alignment paradigm is its versatility; it can be applied at any stage of the machine learning pipeline:
- Pre-processing: Modifying the underlying data or graph structures before training. For instance, Alireza Pirhadi et al. in OTClean: Data Cleaning for Conditional Independence Violations using Optimal Transport use OT to repair datasets that violate conditional independence constraints, shifting the data distribution itself to eliminate bias before a model ever sees it.
- In-processing: Adding alignment penalties directly to the loss function during training. This is widely used in vision-language models, where Yan Luo et al. in FairCLIP: Harnessing Fairness in Vision-Language Learning minimize the Sinkhorn distance between the feature distributions of demographic subgroups to prevent the model from relying on demographic shortcuts.
- Post-processing: Adjusting the outputs of a pre-trained, biased model. By computing optimal transport maps from biased group scores to a fair barycenter, practitioners can guarantee threshold-invariant fairness without needing to alter or retrain the underlying model architecture, as shown by Kweku Kwegyir-Aggrey et al. in Repairing Regressors for Fair Binary Classification at Any Decision Threshold.