Trajectory Regularization via Dynamic Optimal Transport

34 concepts27 papers1 proposal

Continuous-time generative models, such as Continuous Normalizing Flows (CNFs) and Neural Ordinary Differential Equations (Neural ODEs), transform a simple source distribution into a complex target distribution by integrating a learned velocity field over time.

However, standard training objectives—such as maximum likelihood or density matching—suffer from a fundamental blind spot: they only care about the final destination. Because the path itself is unconstrained, the neural network often learns highly convoluted, "wiggly" trajectories.

This lack of constraint creates a massive computational bottleneck. When trajectories are chaotic, numerical ODE solvers are forced to take tiny, computationally expensive steps to maintain accuracy. To solve this, a powerful paradigm has emerged: regularizing continuous-time models by minimizing the kinetic energy of their trajectories. This approach bridges deep learning with fluid dynamics and optimal transport theory, forcing models to learn straight, efficient, and physically grounded paths.

The Intuition: Straightening the Flow

To understand why kinetic energy minimization works, imagine moving a swarm of particles from an initial configuration to a target shape. If the particles wander aimlessly, loop around, or cross paths, they waste energy and make the overall transformation difficult to track.

In physics, the kinetic energy of a particle moving with velocity $v(x,t)$ is proportional to $\|v(x,t)\|^2$. By integrating this squared norm over the entire transformation path, we obtain the total kinetic energy of the system. Minimizing this total energy penalizes unnecessary acceleration and high-velocity maneuvers.

Geometrically, the most energy-efficient path between two states is a straight line traversed at a constant speed—a geodesic. In the space of probability distributions, minimizing this integrated kinetic energy corresponds exactly to the Benamou–Brenier formulation of Optimal Transport (OT). Enforcing this penalty forces the learned velocity field to define a Wasserstein-2 geodesic, ensuring that mass moves along the shortest possible path.

As Xu demonstrates, parameterizing a velocity field with a neural ODE to minimize kinetic energy allows networks to directly discover these optimal trajectories, outperforming static mapping methods on high-dimensional tasks.

Unregularized Flow:   Source (•) ----~~\~~~/\~~~> Target (★)  [Wiggly, slow to solve]
Regularized Flow:     Source (•) -----------------> Target (★)  [Straight, fast to solve]

Slashing Computational Cost

The primary practical benefit of straight trajectories is a dramatic reduction in training and inference times. When a model's paths are straight lines, a numerical solver can take massive steps without losing accuracy, drastically reducing the Number of Function Evaluations (NFE).

The authors of several foundational papers have leveraged this property to optimize continuous models:

  • Neural ODE Acceleration: Finlay et al. showed that introducing a kinetic energy penalty simplifies the integration path, slashing Neural ODE training times by nearly 3x while maintaining generative performance.
  • OT-Flow: Onken et al. combined an $L_2$ transport cost with a Hamilton-Jacobi-Bellman regularization term. By forcing the model to learn straight-line trajectories, they achieved an 8x speedup in training and a 24x speedup in inference.
  • Diffusion ODEs: In the context of diffusion models, Zheng et al. applied second-order regularization to the vector field to enforce straighter paths, improving both likelihood estimation and inference speed.
  • Ascent Dynamics: Ding introduced an ascent regularization that penalizes deviations from theoretical steepest ascent dynamics, guiding the model along simpler latent paths and eliminating the "brittleness" common in unregularized flows.

Alternative Paths to Smoothness

While penalizing the velocity field directly is the most common approach, researchers have developed closely related regularization techniques that target the network's structural complexity to achieve the same minimal-effort trajectories.

Instead of penalizing particle velocity, Dobigeon et al. propose penalizing the energy of the Jacobians across the layers of a neural transport network. This Sobolev-like regularization ensures that the transport effort is distributed equally across transformation steps, preventing chaotic translations. This mirrors the fluid-dynamics perspective of Sobolev Descent proposed by Mroueh et al., where data points move along the gradient flow of a critic to minimize total kinetic energy.

Taking a control-theoretic view, Vialard et al. frame deep learning as a "shooting" problem from optimal control. Rather than regularizing data trajectories, they penalize the kinetic energy of the weight trajectories over time, encouraging the network to find the smoothest possible transformation of the input space.

Mathematical Foundations and Geometry

The transition from static mapping to dynamic trajectory optimization has deep roots in geometric analysis. Historically, optimal transport relied on static elliptic partial differential equations. As Kim et al. note, shifting the problem to dynamic, parabolic flows provides a natural evolutionary path that deforms sub-optimal potential functions into optimal solutions over time.

To apply these principles to real-world data, the underlying geometry must be preserved during discretization:

  • Discrete Surfaces: Lavenant et al. developed a structure-preserving framework by discretizing the Benamou–Brenier formula on triangle meshes, enabling smooth, physically plausible interpolations on curved surfaces without the blurring typical of static methods.
  • Data Assimilation: Pietschmann et al. utilized a primal-dual algorithm based on the Benamou–Brenier dynamic formulation to correct non-linear PDE trajectories using observed data, operating directly within the Wasserstein space of probability measures.

Applications Across Domains

Because kinetic energy minimization produces trajectories that are physically grounded and mathematically unique, this framework has found utility far beyond standard image generation.

Biological Trajectory Inference

In single-cell RNA sequencing, measurements are destructive, meaning researchers can only capture static "snapshots" of different cells at different time points. To reconstruct how individual cells evolve, Tong et al. developed TrajectoryNet. By applying an $L_2$ velocity penalty to a CNF, the model is forced to approximate Wasserstein-2 optimal transport paths. This constraint ensures that the interpolated cellular trajectories follow the shortest, most biologically plausible path rather than arbitrary transformations.

Sampling Multi-Modal Distributions

Sampling from high-dimensional, multi-modal distributions is notoriously difficult because standard normalizing flows often get stuck in local modes. To overcome this, Wu et al. introduced Annealing Flow. By optimizing a dynamic optimal transport objective between a sequence of gradually warming distributions, the model minimizes the movement cost of probability mass between annealing steps. This shortest-path constraint provides a stable training signal and reduces the required intermediate steps by nearly an order of magnitude.

Nonequilibrium Thermodynamics

The mathematical link between optimal transport and physics is not just an analogy; it is an equivalence. Chennakesavalu unified optimal transport theory with nonequilibrium thermodynamics to optimize the "driving protocols" of nanoscale machines. By finding displacement interpolations that minimize kinetic energy, the method identifies the exact manipulation sequence required to move a system between states while losing as little energy as possible to heat.

Causal Inference and Domain Alignment

Dynamic optimal transport also provides rigorous structural guarantees. De Sousa Ribeiro et al. utilized energy-minimizing paths to guarantee the identifiability of high-dimensional counterfactuals (e.g., predicting how a medical image would change under a specific intervention). Because the energy-minimizing path yields a unique mapping that is the gradient of a convex potential, it provides a mathematically sound way to generalize one-dimensional monotonicity to high-dimensional causal data.

Similarly, in cross-domain recommendation systems where user and item spaces do not overlap, Chen et al. constrained embedding transformations to follow a straight, constant-speed path, preventing the noisy and inefficient mappings that typically plague unconstrained neural differential equations.

Key Foundational Work

Minimal Geodesics Along Volume-Preserving Maps, Through Semidiscrete Optimal Transport (2016) — contributed 1 concept elements to this direction.
Parabolic optimal transport equations on manifolds (2010) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_kinetic_energy_minimization_bbc6b027,
  title = {kinetic energy minimization},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/kinetic-energy-minimization/rdr_bbc6b027d1a246a18408fd6c4a52f33c},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-22},
}

This exact version (version 1, frozen text):

@misc{lacuna_kinetic_energy_minimization_bbc6b027_v1,
  title = {kinetic energy minimization},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/kinetic-energy-minimization/rdr_bbc6b027d1a246a18408fd6c4a52f33c/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-22},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake