Optimal Transport (OT) provides a powerful mathematical framework for transforming one probability distribution into another with minimal cost. Historically, however, practical algorithms for solving OT have suffered from a fundamental limitation: they are descriptive rather than predictive.
When you run a classical discrete OT solver (like the Sinkhorn algorithm) on a source dataset $X$ and a target dataset $Y$, it yields a discrete coupling matrix. This matrix tells you exactly how to distribute the mass among the specific data points present during training. But if a new, unseen data point $x_{\text{new}}$ arrives at test time, the discrete plan has no answer. To find where $x_{\text{new}}$ should go, you would have to add it to the dataset and re-solve the entire computationally expensive optimization problem from scratch.
To transition OT from a static optimization tool into a generalized machine learning model, researchers have shifted from learning discrete couplings to estimating continuous transport maps. This allows for immediate out-of-sample extension, turning an $O(N^3)$ optimization problem at test time into a single forward pass through a learned function.
The Core Mechanism: From Couplings to Functions
The fundamental strategy for achieving out-of-sample generalization is to parameterize the transport plan as a continuous function $T: \mathcal{X} \to \mathcal{Y}$. Instead of treating the mapping as an arbitrary permutation of points, the function learns the underlying geometric transformation rules.
An early breakthrough by Michaël Perrot et al. in Mapping Estimation for Discrete Optimal Transport introduced a framework to jointly optimize a discrete coupling matrix and a continuous transformation. By using the discrete coupling to guide the training of a parametric mapping, the authors bridged the gap between discrete samples and continuous spaces, allowing the learned rules to apply directly to unseen data points.
Discrete OT: [Source Points X] --(Coupling Matrix)--> [Target Points Y] (No path for X_new)
Continuous OT: [Source Distribution] ----(Neural Map T(x))----> [Target Distribution]
|
T(x_new) available instantly
Scaling Up with Neural Parameterization
While early methods relied on linear or kernel-based mappings, modern high-dimensional datasets require the expressive capacity of deep neural networks.
Vivien Seguy et al. in Large Scale Optimal Transport and Mapping Estimation proposed a scalable two-step framework. They combined a stochastic dual optimization approach with deep neural networks to learn continuous optimal mappings between massive datasets. Similarly, Nicolas Dobigeon et al. demonstrated in Learning Optimal Transport Between Two Empirical Distributions with Normalizing Flows that normalizing flows can elegantly parameterize these continuous transport maps, ensuring that the underlying geometric rules generalize across entirely different shapes or datasets.
By framing transport as a continuous neural network inference task, models can process arbitrary new samples at test time. This capability is crucial for architectures like CycleGAN, where Guansong Lu in Guiding the One-to-One Mapping in CycleGAN via Optimal Transport utilized OT as a continuous reference map to guide image-to-image translation models toward stable, one-to-one transformations on unseen images.
Injecting Inductive Biases: Geometric Regularization
An unconstrained neural network has too much freedom; it might learn a jagged, chaotic mapping that fits the training points perfectly but generalizes poorly out-of-sample. To fix this, researchers leverage fundamental theorems from OT physics to constrain the function space.
According to Brenier's theorem, when the transport cost is the standard squared Euclidean distance, the unique optimal transport map is the gradient of a convex function, known as the Brenier potential ($\nabla \psi$).
Alexandre d'Aspremont et al. exploited this structure in Regularity as Regularization: Smooth and Strongly Convex Brenier Potentials in Optimal Transport. By explicitly constraining the transport maps to be gradients of smooth and strongly convex functions, the authors created a principled mechanism to evaluate the map at arbitrary new coordinates. This geometric regularization beats the curse of dimensionality and prevents the mapping from becoming discontinuous or jagged.
Following a similar philosophy of structural constraint, Eloi Tanguy et al. in Constrained Approximate Optimal Transport Maps restricted the search space to specific function classes, such as Lipschitz neural networks, proving that bounding the function's regularity directly translates to robust out-of-sample generalization.
Alternative Functional Formulations
Beyond standard Monge maps, out-of-sample extension has been achieved by reframing OT through other continuous mathematical lenses:
- Kernel Embeddings: Jagarlapudi Saketha Nath et al. in Statistical Optimal Transport posed as Learning Kernel Embedding shifted the problem into a Reproducing Kernel Hilbert Space (RKHS). Learning the transport map as a kernel mean embedding yields a dimension-free sample complexity and an inherent functional mapping for out-of-sample data.
- Empirical Risk Minimization (ERM): For dynamic transport problems like the Schrödinger Bridge Problem (SBP), traditional methods rely on discrete Sinkhorn iterations that require post-hoc smoothing. Denis Belomestny et al. in Schrödinger bridge problem via empirical risk minimization bypassed this by deriving a single nonlinear fixed-point equation estimated via ERM, creating a continuous, sample-efficient generative model.
- Information Maximization: Ching-Yao Chuang developed InfoOT: Information Maximizing Optimal Transport, which incorporates global data structure by maximizing mutual information. The author introduced a "conditional projection" method that maps unseen data points smoothly without being distracted by outliers.
Real-World Impact: The Single-Cell Snapshot Problem
The necessity of out-of-sample extension is vividly illustrated in single-cell genomics. To measure a cell's internal molecular state, scientists must destroy its membrane. This creates a "snapshot" problem: you can observe a population of cells at time $t_1$, and a different population at time $t_2$, but you can never track the same cell across time.
Time t_1: [Cell Snapshot A] (Destroyed during measurement)
\
\ Continuous Neural OT Map
\
Time t_2: ---> [Predicted Future State for Unseen Cell]
Discrete OT solvers fail here because they cannot predict the trajectory of a newly sampled cell from a patient. By utilizing continuous neural formulations, biological toolkits can perform true predictive inference.
For instance, Dominik Klein et al. developed the moscot framework in Mapping cells through time and space with moscot to scale optimal transport to millions of cells by transitioning to continuous neural formulations. To handle the fact that cells divide and die, Charlotte Bunne et al. introduced Neural Unbalanced Optimal Transport via Cycle-Consistent Semi-Couplings, which learns continuous, non-linear functions capable of predicting the fate of out-of-sample cells in unbalanced systems. Furthermore, Dominik Klein et al. introduced GENOT: Entropic (Gromov) Wasserstein Flow Matching with Applications to Single-Cell Genomics, leveraging flow matching to provide continuous out-of-sample predictions across high-dimensional biological atlases.
By shifting the core mathematical representation from discrete matrices to continuous functions, these frameworks allow optimal transport to function as a true generalization engine across machine learning and the physical sciences.
Key Foundational Work
Mapping Estimation for Discrete Optimal Transport (2016) — contributed 2 concept elements to this direction.
Large Scale Optimal Transport and Mapping Estimation (2018) — contributed 1 concept elements to this direction.