Velocity Parameterization for Stable Diffusion Trajectories

This direction is split into other directions: velocity-based diffusion objective. Its last version stays readable and citable here.

Version 1 of 122 papers30 conceptsfrozen 2026-10-04

In standard diffusion models, we typically train a network to predict the noise ($\epsilon$) added to an image. While this works well for generating pretty pictures, it encounters a fundamental mathematical breakdown at the boundaries of the diffusion process. As we move toward "Zero Terminal SNR"—the point of pure, featureless noise—the standard noise-prediction objective becomes unstable or even trivial.

Velocity parameterization, often called v-prediction, reformulates the learning target. Instead of predicting the noise or the original data, the model predicts the "velocity" of the transformation along the diffusion path. This shift provides the numerical stability and gradient signal necessary for high-fidelity generation, faster training, and superior generalization to dense prediction tasks.

The Signal-to-Noise Bottleneck

To understand why we need velocity, we must look at the failure modes of standard parameterizations. In a typical diffusion setup, we interpolate between data ($x_0$) and noise ($\epsilon$) to create a noisy sample $x_t$.

If a model is trained to predict $\epsilon$ (noise-prediction), it struggles when the signal-to-noise ratio (SNR) is very high (near the data). Conversely, if it predicts $x_0$ (data-prediction), it struggles when the SNR is very low (near pure noise). As Alex Birch notes in the development of NovelAI Diffusion V3, noise-prediction models suffer from "mean leakage." Because they cannot effectively process the limit where SNR reaches zero, they fail to generate true blacks or pure whites, as the model lacks a useful gradient to move away from a neutral gray starting point.

Defining the Velocity Vector

Velocity parameterization solves this by predicting a weighted combination of the data and the noise. Mathematically, if we define the diffusion path as an interpolation $x_t = \cos(\phi)x_0 + \sin(\phi)\epsilon$, the velocity $v$ is the derivative of this path with respect to the angle $\phi$.

$$v_t = \cos(\phi)\epsilon - \sin(\phi)x_0$$

This formulation, explored deeply by Kaiwen Zheng, Cheng Lu, and Jun Zhu in Improved Techniques for Maximum Likelihood Estimation for Diffusion ODEs, treats the diffusion process as a flow. By predicting the "drift" or rate of change, the model maintains a consistent learning objective across all time steps. It doesn't over-focus on high-noise or low-noise regions; instead, it learns the specific direction and speed required to transform noise into data.

Why It Works: Balancing the Objective

The primary intuition behind v-prediction's success is balance. In Generative Diffusion Modeling: A Practical Handbook, Chi Jin and Zihan Ding explain that velocity prediction provides greater numerical stability because it avoids the mathematical singularities that occur near the data distribution in score-based methods.

This stability manifests in several ways:

  1. Faster Convergence: Kaiwen Zheng and colleagues demonstrated that velocity-based models train 2–3x faster than traditional diffusion models while achieving better density estimation (likelihood).
  2. Numerical Robustness: In low-precision training environments like bfloat16, standard noise prediction often suffers from rounding errors at low SNR. Dacheng Yin and the team behind MMAR found that v-prediction is essential for integrating diffusion into large auto-regressive models, as it prevents gradient instability and model divergence.
  3. Straight-Line Flows: As shown in SiT by Nanye Ma and Michael S. Albergo, velocity prediction is the natural language of "Flow Matching." It allows models to learn straight-line trajectories from noise to data, which are much easier to solve with ODE solvers than the curved paths of standard diffusion.

Beyond Generation: Generalizable Dense Prediction

The benefits of velocity parameterization extend beyond creating new images; they are also crucial for understanding existing ones. In Exploiting Diffusion Prior for Generalizable Pixel-Level Semantic Prediction, Hsin-Ying Lee, Hung-Yu Tseng, and Ming-Hsuan Yang show that v-prediction is superior for dense tasks like depth estimation and semantic segmentation.

By reformulating the diffusion process into a deterministic image-to-prediction mapping, their DMP framework uses the velocity signal to refine structural details. Because the velocity vector captures a more holistic representation of the image's geometry and semantics than raw noise, the model can generalize from a small dataset of bedroom images to predicting the depth of cubist paintings or sci-fi landscapes.

Whether it is Clément Guilloteau using it to generate high-resolution rainfall maps or Amir Barati Farimani reconstructing turbulent fluid flows, velocity parameterization has emerged as the standard for models that require both mathematical rigor and physical consistency across scales.

Papers in this version

History & citationVersion 1 · 2026-10-04

Split into other directions · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_velocity_parameterization_for_stable_dif_94d6d594,
  title = {Velocity Parameterization for Stable Diffusion Trajectories},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/velocity-parameterization-for-stable-diffusion-trajectories/rdr_94d6d5949e4247b8b989e672d9866899},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 1, frozen text):

@misc{lacuna_velocity_parameterization_for_stable_dif_94d6d594_v1,
  title = {Velocity Parameterization for Stable Diffusion Trajectories},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/velocity-parameterization-for-stable-diffusion-trajectories/rdr_94d6d5949e4247b8b989e672d9866899/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}