Generative models like Stable Diffusion have revolutionized AI, but they suffer from a fundamental "speed limit." To transform a block of random noise into a high-fidelity image or audio clip, these models typically perform dozens or even hundreds of iterative denoising steps.
This computational bottleneck stems from the geometry of the paths the models follow. In standard diffusion, the mathematical trajectory from noise to data is highly curved and complex. To navigate these "curvy" paths without drifting off-course, solvers must take many tiny, cautious steps.
A growing body of research, centered on Rectified Flow and Flow Matching, proposes a simple but powerful solution: if we can "straighten" these trajectories into direct, linear paths, we can reach the destination in a fraction of the time.
The Cost of Curvature
In continuous-time generative models, we define a probability path that evolves from a simple distribution (noise) to a complex one (data). As noted by Yixuan Zhu and colleagues in FlowIE, traditional diffusion models follow non-linear paths that necessitate solving complex differential equations over many iterations.
Think of it like driving a car. If the road is a winding mountain pass, you must drive slowly and make constant steering adjustments to stay on the pavement. If you try to take a single giant leap forward, you’ll end up in the ditch. This is why diffusion models require a high Number of Function Evaluations (NFE) to maintain accuracy.
Straightening the Path
Rectified Flow, as explored by Hanshu Yan and Xingchao Liu in PeRFlow, aims to turn that winding road into a straight highway. By targeting straight transport trajectories, the model learns a constant velocity field.
When the trajectory is a straight line, the "steering" (the velocity) doesn't change over time. This allows the solver to take much larger steps—or even a single step—without losing fidelity. For instance, DepthFM, developed by Ming Gui and others, leverages this to achieve sharp monocular depth estimation in as little as a single inference step.
How to Learn a Straight Flow
The challenge is how to force the model to learn these straight paths during training. Two primary mechanisms have emerged:
- Flow Matching (FM): Instead of predicting the "score" (the gradient of the density), models like those proposed by Yaron Lipman and Ricky T. Q. Chen in Multisample Flow Matching learn to regress the velocity field directly.
- Optimal Transport (OT) Couplings: A major cause of "curvy" paths is the random pairing of noise samples with data samples during training. If a noise point on the "left" is paired with a data point on the "right," and vice versa, the paths must cross and bend to avoid each other. Alexander Tong and Nikolay Malkin demonstrated in Conditional Flow Matching that using minibatch Optimal Transport to pair the closest noise and data points "untangles" the trajectories, naturally leading to straighter flows.
Real-Time Generation Across Modalities
The efficiency gains from straightening these flows are transformative across different media:
- Audio: RFWave, by Dongyang Dai and Peng Liu, reconstructs audio waveforms 160x faster than real-time by operating on multi-band complex spectrograms with Rectified Flow.
- Video-to-Audio: The Frieren framework by Wenxiang Guo and Rongjie Huang uses flow matching to achieve high-quality temporal synchronization between video and sound at speeds nearly 10x faster than diffusion baselines.
- 3D Motion: FlowerDance, introduced by Kaixing Yang, generates 3D dance sequences at an unprecedented 2008 FPS by replacing iterative denoising with a streamlined "MeanFlow" strategy.
By viewing generative modeling as a dynamical system governed by differential equations—a perspective synthesized by Lianfang Wang and Jun Liu in Deep Neural Networks Inspired by Differential Equations—researchers have moved from "simulating" noise decay to "engineering" the most efficient path possible between randomness and structure.