Optimal Transport (OT) has become a foundational framework in machine learning for comparing probability distributions. By computing the minimum cost required to morph one distribution into another, metrics like the Wasserstein distance capture the underlying geometry of data spaces in a way that simpler metrics, such as Total Variation or Kullback-Leibler (KL) divergence, cannot.
However, standard optimal transport has a major vulnerability: it is exceptionally sensitive to outliers. A single noisy data point placed far from the true distribution can cause the calculated Wasserstein distance to explode. This vulnerability stems directly from the mathematical constraints of classical OT. To make optimal transport viable for real-world, noisy datasets, a major research direction focuses on modifying these constraints to build outlier-robust transport frameworks.
The Root of Outlier Sensitivity: Mass Conservation
To understand why standard OT fails in the presence of noise, we must look at its defining constraint: strict mass conservation.
In the classical Monge-Kantorovich formulation, we seek a transport plan $\pi$ that aligns a source distribution $\mu$ to a target distribution $\nu$. The optimization is bound by rigid marginal constraints:
$$\sum_j \pi_{ij} = \mu_i \quad \text{and} \quad \sum_i \pi_{ij} = \nu_j$$
This means that every single unit of mass in the source distribution must be accounted for and mapped to a corresponding unit of mass in the target distribution.
As highlighted by Nietert et al. in Outlier-Robust Optimal Transport: Duality, Structure, and Statistical Analysis, because the cost of moving mass scales with the distance between points (often squared distance in $W_2$), extreme outliers exert a disproportionate pull on the optimization. The solver is forced to stretch the transport plan across vast distances to satisfy the marginal constraints for a few anomalous points.
Standard OT (Strict Matching):
Source: [ * * * * ] -----------------------------> Outlier: [ * ] (Massive Cost!)
Robust/Unbalanced OT (Relaxed Matching):
Source: [ * * * * ]
(Outlier ignored or penalized locally via mass destruction)
This sensitivity manifests in several ways:
- Tail Sensitivity: As Chae et al. demonstrate in Posterior asymptotics in Wasserstein metrics on the real line, small amounts of probability mass placed far in the tails can prevent statistical convergence, causing models to overfit to sampling artifacts rather than learning the true underlying signal.
- Geometric Distortion: In tasks like domain adaptation or generative modeling, forcing a model to transport outlier mass warps the learned latent geometry. Chellappa et al. note in Robust Optimal Transport with Applications in Generative Modeling and Domain Adaptation that these strict constraints compel the model to account for bad data, degrading global alignment quality.
To circumvent this, researchers have developed three primary mechanisms: relaxing marginal constraints, statistical trimming, and low-dimensional subspace projections.
Mechanism 1: Relaxing Marginals via Unbalanced and Partial OT
The most direct way to handle outliers is to relax the requirement that all mass must be transported. This gives rise to Unbalanced Optimal Transport (UOT) and Partial Optimal Transport (POT).
Instead of enforcing strict equality on the marginals, UOT introduces a soft penalty—often using KL divergence or Total Variation—that allows for mass creation or destruction. The objective balances the geometric cost of moving mass against the cost of altering the distribution's local mass profile:
$$\min_{\pi} \langle \pi, C \rangle + \lambda_1 \text{D}(\pi_1, \mu) + \lambda_2 \text{D}(\pi_2, \nu)$$
When the solver encounters an outlier far away from the main data clusters, the geometric transport cost $\langle \pi, C \rangle$ becomes larger than the penalty $\lambda$ for violating the marginal constraint. The system chooses to "destroy" or ignore the outlier mass locally rather than paying the massive cost to transport it across the space.
Le et al. show in On Robust Optimal Transport: Computational Complexity and Barycenter Computation that these relaxed formulations are not only more resilient to noise, but they are also computationally faster to solve than standard OT because they avoid the rigid bottlenecks of exact matching. Similarly, Fatras et al. leverage this property in Unbalanced minibatch Optimal Transport; applications to Domain Adaptation to prevent mini-batch sampling noise from distorting global domain alignments.
In contrast to the soft penalties of UOT, Partial OT explicitly permits discarding a fixed fraction of the total mass. Pouyan Shirzadian introduces a framework in A New Robust Partial p-Wasserstein-Based Metric for Comparing Distributions that bounds the impact of extreme values by isolating a core percentage of the distribution, ensuring stable distance measurements even in heavy-tailed settings.
Mechanism 2: Robust Statistical Filtering and Trimming
Rather than altering the transport objective itself, another paradigm integrates classical robust statistics directly into the data-handling step of the optimal transport pipeline.
One approach is Data Depth Trimming. In A Pseudo-Metric between Probability Distributions based on Depth-Trimmed Regions, Staerman et al. utilize multivariate data depth to generalize quantiles to high-dimensional spaces. By evaluating transport distances only between the "deepest" regions of the distributions, the outer layers—where outliers reside—are systematically trimmed away before the transport plan is ever computed.
Alternatively, the Median-of-Means (MoM) framework can be applied to transport problems. Staerman et al. demonstrate in When OT meets MoM: Robust estimation of Wasserstein Distance that by partitioning a dataset into independent blocks, calculating the optimal transport distance for each block, and taking the median of those distances, the influence of adversarial or corrupted samples is naturally neutralized. Because outliers are isolated to a few blocks, they cannot skew the global median estimate.
Mechanism 3: Low-Dimensional Subspace Projections
In high-dimensional settings, random noise and outliers can accumulate, obscuring the true structural differences between distributions. To combat this, researchers project the data onto informative, low-dimensional subspaces where the underlying geometry is preserved but noise is filtered out.
Paty et al. introduce this concept in Subspace Robust Wasserstein Distances. The core intuition is to search for a low-dimensional subspace that maximizes the Wasserstein distance between two distributions:
$$\max_{E \in \mathcal{G}_k} W_p(P_E \mu, P_E \nu)$$
By maximizing the distance over a rank-constrained projection matrix, the model actively ignores orthogonal dimensions that contain only random sampling artifacts or isolated outliers, focusing instead on the principal geometric displacements between the core masses.
To scale this intuition to massive datasets, Bonet et al. combine projection methods with unbalanced transport in Slicing Unbalanced Optimal Transport. By projecting high-dimensional distributions onto random 1D lines (slicing) and solving the robust, unbalanced transport problem in one dimension, they achieve both computational efficiency and resilience against extreme anomalies.
Downstream Impact: Robust Barycenters and Maps
Fixing the outlier sensitivity of optimal transport fundamentally changes the behavior of downstream applications, most notably in computing Wasserstein Barycenters (the geometric average of multiple distributions).
When averaging distributions using standard OT, even a small percentage of stray data points in one source distribution will pull the barycenter toward the noise. By replacing standard transport with robust variants:
- Cazelles et al. develop weak barycenters in A novel notion of barycenter for probability distributions based on optimal weak mass transport to extract shared latent structures while ignoring distribution-specific noise.
- Gazdieva et al. show in Robust Barycenter Estimation using Semi-Unbalanced Neural Optimal Transport that semi-unbalanced formulations allow continuous neural estimators to average complex data sources while filtering out class imbalances and corrupted samples.
Ultimately, mitigating outlier sensitivity transforms optimal transport from a brittle, theoretical metric into a highly stable, practical tool capable of operating reliably on messy, real-world data.
Key Foundational Work
Subspace Robust Wasserstein Distances (2019) — contributed 1 concept elements to this direction.
Statistical Optimal Transport via Factored Couplings (2019) — contributed 1 concept elements to this direction.
A Simulated Annealing Based Inexact Oracle for Wasserstein Loss Minimization (2017) — contributed 1 concept elements to this direction.