Learnable Forward Processes for Diffusion Models

This direction is merged into another direction: learnable diffusion coefficients. Its last version stays readable and citable here.

Version 1 of 116 papers22 conceptsfrozen 2026-10-04

Standard diffusion models operate on a fixed premise: we know exactly how to destroy information. By following a predefined schedule—usually a linear or cosine ramp of Gaussian noise—we transform structured data into a spherical distribution. The model's only job is to learn the reverse.

However, a growing body of research suggests that this "fixed" forward process is a bottleneck. By treating the noising process as a learnable component rather than a static heuristic, researchers have found they can significantly accelerate sampling, improve likelihoods, and adapt models to complex data geometries.

The Bottleneck of Fixed Schedules

In traditional diffusion, the forward path is data-agnostic. Whether you are noising a picture of a cat or a complex molecular graph, the noise is injected the same way. This rigidity often forces the reverse (generative) process to learn highly curved, complex trajectories to recover the data.

Grigory Bartosh, Christian Andersson Naesseth, and colleagues argue in Neural Flow Diffusion Models that these rigid structures prevent models from achieving tighter variational bounds. If the forward process doesn't "fit" the data's natural structure, the reverse process must work harder, requiring hundreds or thousands of small steps to stay on the correct manifold during generation.

Bilateral Optimization: Learning to Noise

The core shift in this research direction is the move toward Bilateral Denoising Diffusion Models (BDDM). Instead of just training a denoiser, these models use an auxiliary "schedule network" to optimize the noise injection itself.

Max W. Y. Lam, Dan Su, and Rongjie Huang demonstrated this in Bilateral Denoising Diffusion Models. By parameterizing the forward process, the model learns to select the most "informative" steps. In speech synthesis, this allows FastDiff to reduce the number of iterations from a thousand to as few as four, while maintaining high audio quality.

The intuition here is simple: if you can choose how to "blur" an image, you can do it in a way that makes "unblurring" it much easier. By optimizing the noise ratios for a specific number of steps, the model finds a more efficient path through the latent space.

Adapting to Data Geometry

For data with specific physical or mathematical constraints, learnable forward processes are even more critical. In Equivariant Neural Diffusion for Molecule Generation, François R. J. Cornet and Grigory Bartosh show that a learnable, data-dependent transformation can respect 3D symmetries (equivariance) better than fixed noise.

Similarly, Weitao Du and colleagues introduced FP-Diffusion, which uses Riemannian and symplectic geometry to parameterize the forward stochastic differential equation (SDE). This allows the diffusion path to adapt to the underlying manifold of the data. Instead of forcing the data into a standard Gaussian "box," the model learns a forward path that preserves the natural geometric relationships of the input.

Why It Works: Path Simplification

The ultimate goal of learning the forward process is path simplification.

In Forward-Learned Discrete Diffusion (FLDD), Grigory Bartosh and Teodora Pandeva explain that by conditioning the noise distribution on the entire input, the model discovers latent trajectories that are inherently easier to reverse.

Think of it like a mountain path. A fixed schedule is like a straight line drawn on a map—it might be the shortest distance, but it could require climbing over impossible cliffs. A learnable forward process is like a scout finding a winding trail that follows the terrain. The trail is longer in "distance," but much faster and easier to traverse. By "learning how to noise," these models ensure the generative "climb" back to high-fidelity data is as smooth and direct as possible.

Papers in this version

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_learnable_forward_processes_for_diffusio_06654f3d,
  title = {Learnable Forward Processes for Diffusion Models},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/learnable-forward-processes-for-diffusion-models/rdr_06654f3dd01d4e42b206a5a0a000f149},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 1, frozen text):

@misc{lacuna_learnable_forward_processes_for_diffusio_06654f3d_v1,
  title = {Learnable Forward Processes for Diffusion Models},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/learnable-forward-processes-for-diffusion-models/rdr_06654f3dd01d4e42b206a5a0a000f149/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}