Minimax Optimization for Neural Optimal Transport

38 concepts34 papers

Optimal Transport (OT) provides a mathematically rigorous framework for comparing and transforming probability distributions. It answers a fundamental question: what is the most efficient way to reshape a source distribution (e.g., a prior noise distribution or an image domain) into a target distribution?

Historically, computing this optimal transformation required solving the discrete Monge or Kantorovich problems. These classical approaches suffer from severe computational bottlenecks, scaling cubically ($O(N^3)$) with the number of data points, or relying on mini-batch approximations that introduce significant bias, as noted by Mengxue Li et al. in Enhanced Transport Distance for Unsupervised Domain Adaptation. To scale OT to high-dimensional, continuous machine learning tasks like image generation and point cloud processing, researchers have turned to neural potential optimization. This paradigm reformulates the optimal transport problem into a continuous minimax (saddle-point) game, parameterizing the underlying mathematical potentials and transport maps with deep neural networks.

The Core Intuition: From Primal Matching to Dual Potentials

To understand why this approach works, we must look at the transition from the primal formulation of optimal transport to its dual representation.

In the primal formulation (the Kantorovich problem), we search for a joint distribution or "transport plan" $\pi(x, y)$ that minimizes the expected transport cost $\mathbb{E}_{(x,y)\sim\pi}[c(x,y)]$, subject to the constraint that the marginals of $\pi$ must exactly match our source and target distributions. When dealing with continuous, high-dimensional spaces, tracking every possible pairing between points becomes intractable.

Kantorovich duality flips this perspective. Instead of optimizing over a massive joint distribution space, the dual formulation optimizes over two scalar functions, $\phi(x)$ and $\psi(y)$, known as Kantorovich potentials. Intuitively, these potentials act like a dynamic pricing or pressure field across the domains. The dual objective maximizes the expected "profit" of moving mass:

$$\max_{\phi, \psi} \mathbb{E}_{x \sim \alpha}[\phi(x)] + \mathbb{E}_{y \sim \beta}[\psi(y)] \quad \text{subject to} \quad \phi(x) + \psi(y) \le c(x,y)$$

The constraint ensures that the price difference between two locations never exceeds the actual physical cost $c(x,y)$ of moving mass between them. At optimality, the constraint holds with equality precisely along the paths where mass should move.

By parameterizing these potentials as continuous neural networks, we completely bypass the need for discrete grids or histograms. We can evaluate the potentials at any arbitrary point sampled from our distributions, allowing the model to generalize across continuous space.

The Semi-Dual Formulation and the Minimax Game

While the pure dual formulation eliminates discrete binning, enforcing the pointwise constraint $\phi(x) + \psi(y) \le c(x,y)$ across infinite continuous points is challenging. Early neural OT methods addressed this by enforcing structural constraints on the networks, such as using Input Convex Neural Networks (ICNNs) to satisfy Brenier's theorem, as explored in Optimal transport mapping via input convex neural networks.

However, a more flexible and widely adopted strategy is the semi-dual formulation. By leveraging the mathematical concept of the $c$-transform, one potential can be implicitly defined in terms of the other: $\phi(x) = \inf_{y} [c(x,y) - \psi(y)]$. Substituting this back into the objective transforms the problem into an unconstrained minimax optimization.

In practice, this manifests as an adversarial game between two neural networks:

  1. The Potential Network ($\psi$): Acts as a discriminator or critic, learning the geometry of the target domain and enforcing the optimality conditions.
  2. The Transport Network ($T$ or $G$): Acts as a generator or "mover," learning the explicit push-forward map that translates source samples directly into the target domain.
[Source Sample x] ---> ( Transport Network T ) ---> [Predicted Target T(x)]
                                                            |
                                                   ( Potential Network ψ )
                                                            |
                                                   [Optimality Penalty]

Jiaojiao Fan et al. demonstrate in Neural Monge Map estimation and its applications that this saddle-point formulation allows for the direct estimation of optimal transport maps without requiring rigid architectural constraints like input convexity. Similarly, Litu Rout leverages this saddle-point objective in Generative Modeling with Optimal Transport Maps to directly minimize the quadratic Wasserstein-2 distance for generative tasks, showing that the generator itself converges to the mathematically optimal transport map.

This adversarial framework differs crucially from standard Generative Adversarial Networks (GANs). In a standard GAN, the discriminator's objective can become a moving target, leading to training instability. In neural OT, the potential network is strictly bound by the cost function $c(x,y)$. This mathematical grounding provides strong convergence properties. Indeed, Roman Tarasov et al. establish the first formal statistical learning bounds for these architectures in A Statistical Learning Perspective on Semi-dual Adversarial Neural Optimal Transport Solvers, proving that given sufficient data and network capacity, the learned transport map reliably converges to the true theoretical Monge map.

Relaxing Constraints: Unbalanced and Partial Transport

In many real-world applications, the assumption of strict mass conservation—that the total mass of the source must exactly equal the target—breaks down. For instance, in unpaired point cloud completion or noisy domain adaptation, datasets frequently exhibit class imbalances, noise, or only partial overlap.

To handle these scenarios, researchers extend the minimax potential framework to Unbalanced Optimal Transport (UOT) and Partial Optimal Transport. Instead of enforcing hard marginal constraints, UOT relaxes them by penalizing the discrepancy between the transported mass and the target marginal using Kullback-Leibler (KL) divergence or other scaling functionals.

Jaemoo Choi et al. show in Generative Modeling through the Semi-dual Formulation of Unbalanced Optimal Transport that the semi-dual formulation of UOT simplifies the required architecture down to just two networks (a potential and a generator), eliminating the need for complex auxiliary networks. This relaxation makes the optimization significantly more robust to outliers and accelerates training. This principle is successfully applied to 3D data by Taekyung Lee et al. in Unpaired Point Cloud Completion using Unbalanced Optimal Transport Map, where the generator learns to complete missing geometries without being distorted by real-world scanning imbalances. Similarly, for rigid and non-rigid alignments, Ziming Wang et al. derive a dual form for partial Wasserstein distance in Partial Wasserstein Adversarial Network for Non-rigid Point Set Registration, allowing a neural potential to automatically isolate and match only the relevant, overlapping regions of two point sets.

Scaling Up: Barycenters and Population Dynamics

The power of parameterizing Kantorovich potentials continuously becomes even more apparent when moving beyond simple one-to-one domain mappings into multi-distribution aggregation and temporal dynamics.

Continuous Wasserstein Barycenters

A Wasserstein barycenter is the "geometric average" of multiple probability distributions. Computing this average traditionally requires a nested, multi-level optimization loop that is notoriously slow. By parameterizing the dual potentials with continuous neural networks, the problem can be flattened.

The authors of Continuous Regularized Wasserstein Barycenters demonstrate that neural dual potentials allow for the aggregation of high-dimensional distributions without the memory constraints of discrete grids, enabling continuous sampling from the estimated central barycenter. To make this robust against real-world data corruption, Milena Gazdieva et al. introduce a semi-unbalanced neural OT barycenter framework (Robust Barycenter Estimation using Semi-unbalanced Neural Optimal Transport), which uses the potential networks to automatically identify and filter out outlier distributions during the averaging process.

Learning Dynamics via JKO Schemes

In population dynamics (such as modeling cell differentiation or crowd movement over time), systems are often governed by the Jordan-Kinderlehrer-Otto (JKO) scheme. The JKO scheme posits that a population evolves by taking successive optimal transport steps to minimize an underlying energy functional—effectively performing gradient descent on the space of probability distributions.

Neural potential optimization allows researchers to scale these models linearly. Jaemoo Choi et al. leverage the mathematical equivalence between JKO steps and unbalanced optimal transport in Scalable Wasserstein Gradient Flow for Generative Modeling through Unbalanced Optimal Transport, converting the discrete time-stepping problem into a sequence of scalable minimax objectives.

Taking this a step further, Mikhail Persiianov et al. introduce iJKOnet in Learning of Population Dynamics: Inverse Optimization Meets JKO Scheme. Here, the minimax framework is used for inverse optimization: an inner neural network agent computes the optimal transport map for a given candidate energy function, while an outer loop adjusts the energy parameters until the predicted transport matches observed real-world time-lapse snapshots.

Summary

Neural potential optimization fundamentally changes how we compute optimal transport. By shifting from the primal problem of matching individual data points to the dual problem of optimizing continuous potential fields, it transforms a combinatorial nightmare into a highly scalable, stable minimax game. Whether enforcing strict mass conservation or relaxing constraints via unbalanced transport, this framework provides a mathematically rigorous yet practical toolkit for deep learning across continuous domains.

Key Foundational Work

On Scalable and Efficient Computation of Large Scale Optimal Transport (2019) — contributed 1 concept elements to this direction.
Scalable Unbalanced Optimal Transport using Generative Adversarial Networks (2019) — contributed 1 concept elements to this direction.
Three-Player Wasserstein GAN via Amortised Duality (2019) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_dual_potential_optimization_80401b44,
  title = {dual potential optimization},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/dual-potential-optimization/rdr_80401b44dc504c54a6d273e9899fa67f},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 1, frozen text):

@misc{lacuna_dual_potential_optimization_80401b44_v1,
  title = {dual potential optimization},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/dual-potential-optimization/rdr_80401b44dc504c54a6d273e9899fa67f/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}

See something wrong? Report a mistake