Generative modeling has largely been dominated by two powerful paradigms: Normalizing Flows and Diffusion Models. Normalizing flows use Ordinary Differential Equations (ODEs) to define a clean, deterministic mapping between a simple base distribution (like Gaussian noise) and a complex target distribution (like natural images). However, training them typically requires Maximum Likelihood Estimation (MLE), which involves computationally expensive and unstable backpropagation through ODE solvers.
On the other hand, Diffusion Models use Stochastic Differential Equations (SDEs) and score matching to achieve simulation-free training and high-quality samples. Yet, they are mathematically complex, theoretically require infinite time to converge, and lack the straightforward deterministic mapping of flows.
This paper introduces InterFlow, a novel framework that bridges this gap. By leveraging "stochastic interpolants," InterFlow retains the deterministic, invertible nature of normalizing flows while adopting the highly efficient, simulation-free training style of diffusion models.
The Core Idea: Stochastic Interpolants
Instead of attempting to learn the complex mapping directly via MLE, the authors propose defining a time-dependent path—an interpolant $I_t(x_0, x_1)$—that explicitly connects a sample from the base distribution ($x_0$) to a sample from the target ($x_1$) over a finite time interval $t \in [0, 1]$.
By decoupling the path from the training objective, the user can choose the interpolant, and the model simply learns the velocity field required to follow it. The authors prove mathematically that the velocity field of this flow is the unique minimizer of a simple quadratic loss function. Because this loss relies only on expectations over the interpolant, the model can be trained without ever solving an ODE during the training loop.
Why InterFlow Stands Out
The InterFlow framework introduces several key technical advantages that solve long-standing bottlenecks in generative modeling:
- Simulation-Free Training: Because the quadratic loss avoids ODE solvers during training, the computational cost per iteration is constant. This leads to massive speedups compared to traditional continuous normalizing flows.
- Finite Time Convergence: Unlike standard diffusion models that theoretically require infinite time to reach the target distribution, InterFlow connects the distributions exactly within the finite interval $t \in [0, 1]$.
- Optimal Transport: The framework allows for the interpolant itself to be optimized. By maximizing the objective over the choice of path, the model can discover "optimal transport" maps—the most efficient, straight-line paths between distributions.
- Duality with Diffusion: If the base distribution is Gaussian, the learned velocity field can be directly converted into a "score" function for a diffusion model, elegantly bridging the ODE and SDE perspectives.
Key Results and Performance
The authors evaluated InterFlow across 2D toy datasets, high-dimensional tabular data, and image generation benchmarks, demonstrating significant improvements in both speed and scalability:
- Massive Speedups: On the MiniBooNE tabular dataset, InterFlow achieved a staggering 400x speedup in training time compared to the state-of-the-art FFJORD model, while simultaneously reaching a better log-likelihood.
- Superior Density Estimation: On standard tabular benchmarks, InterFlow matched or surpassed conventional continuous flows. For instance, on the GAS dataset, it achieved a negative log-likelihood (NLL) of -12.35, easily beating FFJORD's -8.59.
- Scaling to High Resolutions: Historically, ODE-based flows have struggled to scale due to the memory demands of the adjoint method (backpropagating through solvers). By bypassing this entirely, InterFlow successfully generated $128\times128$ images of flowers, proving that ODE flows can reach previously unattainable resolutions.
- Competitive Image Quality: On CIFAR-10, InterFlow achieved an NLL of 2.99 bits/dim, making it highly competitive with top-tier diffusion models (like ScoreSDE) and superior to many previous flow models.
Limitations and Future Directions
While InterFlow represents a major step forward, some open questions remain. Most experiments in the paper utilize a fixed linear or trigonometric path; finding the absolute "best" class of interpolants for specific data modalities is an area for future research. Additionally, while training is simulation-free and incredibly fast, sampling still requires solving an ODE at inference time, which can be slower than single-step models like GANs.
Finally, the authors note that while their ODE approach is simpler and highly effective, the exact role and necessity of stochasticity (noise) in high-dimensional image generation is still being explored. Future work will likely investigate hybrid models that use the stochastic interpolant framework to blend the best aspects of SDEs and ODEs even further.
Summary updated