Optimal transport flow and infinitesimal density ratio estimation

Chen Xu, Xiuyuan Cheng, Yao Xie

Introduction

Optimal Transport (OT) provides a powerful mathematical framework for transforming one probability distribution into another with minimal cost. While OT is fundamental to generative modeling, domain adaptation, and statistical inference, computing it efficiently in high-dimensional spaces remains a significant challenge. Traditional neural approaches often focus on static maps, which can fail to capture the underlying dynamics and geometric structure of the data. This paper introduces Q-flow, a framework that leverages Neural Ordinary Differential Equations (Neural ODEs) to solve the dynamic OT problem, and a corresponding flow-ratio net for robust density ratio estimation (DRE).

Methods

The proposed approach shifts from finding a static mapping to learning a continuous velocity field $v(x, t)$ that transports mass over time.

Q-flow: The Transport Network

Q-flow parameterizes the velocity field using a Neural ODE. The model is trained by minimizing the Benamou-Brenier functional, which seeks a path that both reaches the target distribution and minimizes the total kinetic energy (the $W_2$ transport cost).

A key innovation is bi-directional training. By simultaneously learning the forward ($P \to Q$) and backward ($Q \to P$) flows, the model exploits the inherent symmetry of optimal transport. This dual training serves as a strong regularizer, particularly beneficial when working with finite samples in high dimensions. Unlike "Flow Matching" techniques, Q-flow does not require pre-computed couplings or a fixed reference distribution like a Gaussian.

Flow-ratio Net: Infinitesimal Density Ratio Estimation

Standard DRE methods often struggle when distributions $P$ and $Q$ have little overlap. Q-flow addresses this by creating a smooth "bridge" of intermediate distributions. The flow-ratio net estimates the infinitesimal change in density along the optimal transport path. By integrating these local changes, the model can accurately compute the global density ratio between highly dissimilar distributions.

Illustration of the dynamic optimal transport flow and infinitesimal density ratio estimation.

Experiments and Results

The authors evaluated Q-flow across synthetic benchmarks, image-to-image translation, and generative modeling tasks.

High-Dimensional Benchmarks

On Gaussian mixtures in up to 256 dimensions, Q-flow demonstrated superior accuracy compared to baselines like OTCFM and MMv2, achieving lower $L^2$ Unexplained Variance Percentage (UVP) and higher cosine similarity.

Image-to-Image Translation

In the Handbag $\to$ Shoes task, Q-flow achieved a Fréchet Inception Distance (FID) of 12.34, outperforming CycleGAN (16.00) and other neural OT solvers. This highlights the model's ability to capture complex data geometries in high-dimensional image spaces.

Density Ratio Estimation

Using MNIST images for energy-based model evaluation, the flow-ratio net achieved a bits-per-dimension (BPD) of 1.05, indicating high efficiency in capturing the underlying data distribution compared to previous state-of-the-art telescopic DRE methods.

Key Insights

  1. The Telescopic Advantage: By breaking down a large distributional gap into infinitesimal steps along an OT path, the model avoids the "canyon-jumping" problem of standard DRE.
  2. Geometric Awareness: OT-based paths follow the natural manifold of the data, leading to more accurate interpolation and density estimation than simple linear methods.
  3. Symmetry and Efficiency: Bi-directional flow training improves robustness. Despite the use of Neural ODEs, Q-flow showed competitive training times, converging in approximately 8 hours on a single A100 GPU.

Conclusion

Q-flow and the flow-ratio net provide a robust framework for high-dimensional optimal transport and density ratio estimation. By focusing on the dynamic path and infinitesimal changes, the method overcomes limitations of static mappings and standard DRE, offering a principled approach for complex data transformation tasks. Future work may explore adaptive time grids to further optimize the computational efficiency of the underlying ODE solvers.

Summary updated

BibTeX
@inproceedings{xu2025optimal,
  author = {Chen Xu and Xiuyuan Cheng and Yao Xie},
  title = {{Optimal transport flow and infinitesimal density ratio estimation}},
  booktitle = {International Conference on Artificial Intelligence and Statistics},
  year = {2025},
  url = {https://openreview.net/forum?id=oEWYNesvRJ},
}
Figure