Abstract
We propose dynamical optimal transport (OT) problems constrained in a parameterized probability subset. In application problems such as deep learning, the probability distribution is often generated by a parameterized mapping function. In this case, we derive a simple formulation for the constrained dynamical OT.
Introduction
Optimal Transport (OT) has become a cornerstone of modern machine learning, particularly for tasks involving generative modeling and distribution alignment. The dynamical formulation of OT, known as the Benamou-Brenier problem, interprets the transport of probability mass as a fluid dynamics problem, seeking a velocity field that minimizes kinetic energy. However, a significant gap exists between this theoretical framework and practical applications: while OT operates on the infinite-dimensional space of all probability densities, machine learning models are typically restricted to finite-dimensional parameterized families, such as those defined by neural networks.
The Challenge of Parametric Constraints
In practice, we do not manipulate arbitrary densities. Instead, we work with distributions generated by a mapping function $g_\theta(z)$, where $\theta \in \Theta$ represents the model parameters and $z$ is a latent variable. Standard OT does not inherently respect these parametric constraints. This paper addresses a fundamental question: how can we define and compute the most efficient path between two distributions when we are restricted to a specific manifold of probability measures?
Methodology: Bridging Eulerian and Lagrangian Perspectives
The authors bridge two classical perspectives in fluid dynamics to solve this constrained problem:
- Eulerian Coordinates: This view focuses on the "field"—how the density at a fixed point in space changes over time. The authors define the constrained OT problem here using a "constrained continuity equation."
- Lagrangian Coordinates: This view follows the "particles"—tracking the actual trajectories of individual samples as they move through space.
The core contribution is Proposition 1, where the authors prove that the constrained dynamical OT problem has a simple Lagrangian formulation. By assuming the probability distribution is generated by the mapping function $g_\theta(z)$, they transform a complex optimization problem over abstract density functions into a more manageable problem over the parameter space $\Theta$.
Key Insights
Parameterized Particles
The authors introduce the concept of viewing the generative model $g_\theta(z)$ as a "parameterized particle." Instead of moving every possible point in space independently, the framework only moves the points that the model is capable of representing. This effectively reduces the dimensionality of the problem to the size of the parameter vector $\theta$.
The Metric Tensor and Information Geometry
The paper demonstrates that this constrained OT naturally defines a "geometric action energy" in the parameter space. They derive a metric tensor $G(\theta)$ that describes the "shape" of the parameter manifold. This is deeply related to Information Geometry and the concept of "natural gradients," providing a rigorous link between optimal transport and the geometry of statistical models.
Energy Extensions
The framework is not limited to kinetic energy. The authors show it can incorporate potential energies (representing external costs or landscapes) and interaction energies (representing how different parts of the distribution repel or attract each other). This flexibility allows the model to describe complex physical or social systems within a constrained parametric space.
Results and Implications
The primary result is the derivation of the Lagrangian formulation for constrained OT (Equation 5). This derivation proves that the optimal path in the density space corresponds to a specific path in the parameter space governed by a Hamiltonian function.
For the specific case of the $L^2$-Wasserstein distance, the authors show that the squared distance is the infimum of the kinetic energy of the parameters, subject to a constraint involving the gradient of a potential function $\Phi$. This provides a theoretical foundation for "Wasserstein-like" training in models where one can only update parameters, not the raw densities themselves.
Limitations and Future Work
As a theoretical note, the paper maintains a high level of abstraction. The derivations assume that the mappings and densities are smooth ($C^\infty$), which may not hold for all neural network architectures, such as those using ReLU activations. Furthermore, while the Lagrangian formulation is mathematically simpler, the paper does not provide a specific algorithm for large-scale implementation.
Future research may apply this formulation to improve the stability and efficiency of Generative Adversarial Networks (GANs) and Mean-Field Games, potentially leading to more "geometry-aware" optimization laws in deep learning.
Summary updated