Traditional diffusion models operate by transforming clean data into pure noise, and then learning to reverse this process. During training, a neural network is typically tasked with predicting one of two endpoints: either the original clean data $x_0$ or the random noise $\epsilon$ that was added to it.
While intuitive, this binary choice introduces severe numerical and optimization challenges at the boundaries of the diffusion process. If the model predicts noise, it struggles at the beginning of generation when the input is almost pure noise, because the signal-to-noise ratio (SNR) approaches zero. If it predicts the clean data, the loss landscape becomes highly unstable near the data distribution.
To resolve this dilemma, a powerful alternative has emerged: velocity prediction (often called $v$-prediction or velocity parameterization). Instead of predicting the static endpoints of the transformation, the model is trained to predict the velocity vector—the instantaneous direction and speed of the trajectory moving from noise to data.
The Geometry of the Velocity Target
To understand why velocity prediction works, consider how a data point $x_0$ is mixed with noise $\epsilon$ over time $t$. The intermediate latent state $x_t$ is typically expressed as a linear interpolation:
$$x_t = \alpha_t x_0 + \sigma_t \epsilon$$
Where $\alpha_t$ and $\sigma_t$ dictate the noise schedule. The velocity $v_t$ is simply the time-derivative of this path:
$$v_t = \frac{dx_t}{dt} = \dot{\alpha}_t x_0 + \dot{\sigma}_t \epsilon$$
This formulation reveals the core intuition behind the approach: velocity is a smoothly changing, weighted combination of both the data and the noise.
As originally shown by Salimans et al. and Ho in Progressive Distillation for Fast Sampling of Diffusion Models, predicting this velocity vector provides a unified target that gracefully transitions between noise-prediction and data-prediction depending on the timestep. Rather than forcing the network to make a hard conceptual pivot across different stages of the generation process, velocity parameterization treats the entire trajectory as a continuous flow field.
Eliminating Boundary Singularities
The most compelling mathematical argument for velocity prediction lies in how it behaves at the extreme edges of the diffusion process.
The Zero-SNR Problem
Standard diffusion models often use noise schedules that never fully reach a signal-to-noise ratio of zero. As exposed by Liu et al. and Lin in Common Diffusion Noise Schedules and Sample Steps are Flawed, this design flaw prevents models from generating images with full dynamic range, making truly dark or bright scenes impossible to synthesize. However, if you enforce a "zero terminal SNR" schedule using standard noise prediction, the objective blows up mathematically because the model tries to predict noise from pure noise, creating a singularity. Switching to a velocity target resolves this completely: when the signal is entirely absent, the velocity vector remains bounded and stable, pointing cleanly toward the data distribution.
The Data-Distribution Singularity
A reciprocal issue occurs at the other end of the timeline ($t \to 0$), where the latent state is nearly clean data. Score-based methods often encounter mathematical singularities here because the score function (the gradient of the log-density) diverges as the distribution sharpens. As explored by Ma et al. et al. with Scalable Interpolant Transformers in SiT, velocity-based objectives bypass these singularities entirely. The velocity remains well-behaved throughout the entire transition, ensuring stable training from the first step to the last.
Balancing the Optimization Landscape
In standard diffusion training, the loss function can be highly unbalanced. At certain timesteps, predicting the target is trivial; at others, it is incredibly difficult. This imbalance forces the network to over-allocate its capacity to specific noise regimes while neglecting others.
By reparameterizing the network to target velocity, the learning objective is naturally balanced across all noise levels. In Improved Techniques for Maximum Likelihood Estimation for Diffusion ODEs, Zheng et al. et al. demonstrated that this variance-balancing property prevents the model from over-focusing on regions with extreme SNRs. Consequently, models trained with velocity parameterization converge 2 to 3 times faster than traditional methods while achieving significantly better density estimation accuracy.
Accelerating Inference and Distillation
Beyond training stability, modeling the velocity field yields smoother trajectories. When the underlying vector field is smooth, numerical ordinary differential equation (ODE) solvers can take much larger steps without drifting off course. Zhang et al. and Chen in Fast Sampling of Diffusion Models with Exponential Integrator note that a smoother objective over the sampling trajectory directly facilitates more accurate higher-order numerical approximations, enabling high-quality generation in fewer steps.
This smoothness is particularly vital for model distillation, where a pre-trained model is compressed into a fast, few-step sampler. When a model is forced to make massive updates in a single step, pixel-space or noise-space errors can compound catastrophically.
Standard Space (Noise/Pixel):
[Pure Noise] ----(Large Step Error)----> [Distorted Latent] ----> [Degraded Image]
Velocity Space:
[Pure Noise] ----(Stable Direction)----> [Smooth Trajectory] ----> [High-Fidelity Image]
Distilling in velocity space focuses the student network on learning the overall direction and speed of the flow field rather than direct pixel reconstruction. As demonstrated by Cai et al. et al. in Shortcutting Pre-trained Flow Matching Diffusion Models is Almost Free Lunch, velocity-space distillation provides superior numerical stability and faster convergence because the magnitude of errors allows for more effective gradient updates than the tiny, fractional errors found in pixel space.
Broadening Applications: From Particles to Motion
The conceptual shift from "predicting noise" to "modeling a velocity field" has allowed researchers to apply diffusion mechanics to domains far beyond image synthesis.
- Physical Trajectories: In Hamiltonian Score Matching and Generative Flows, Holderrieth et al. et al. explicitly bridged generative modeling with classical mechanics. By treating data points as physical particles with position and velocity, they developed Hamiltonian Generative Flows, using velocity as a steering mechanism through a continuous vector field.
- Human Motion Generation: For temporal sequences like human motion, predicting noise often leads to jittery, unrealistic movements. Meng et al. et al. demonstrated in both Absolute Coordinates Make Motion Generation Easy and Rethinking Diffusion for Text-Driven Human Motion Generation that velocity-based objectives (derived from flow-matching) yield significantly higher motion fidelity and temporal smoothness.
- Dense Spatial Prediction: Even when repurposing pre-trained diffusion models for downstream tasks like depth estimation or semantic segmentation, Lee et al. et al. found in Exploiting Diffusion Prior for Generalizable Dense Prediction that a velocity signal provides a more effective training target for recovering pixel-level geometric and semantic properties than standard noise prediction.
Key Foundational Work
Progressive Distillation for Fast Sampling of Diffusion Models (2022) — contributed 2 concept elements to this direction.
Improved Techniques for Maximum Likelihood Estimation for Diffusion ODEs (2023) — contributed 2 concept elements to this direction.
Fast Sampling of Diffusion Models with Exponential Integrator (2023) — contributed 1 concept elements to this direction.