Generative Diffusion as Stochastic Optimal Control

116 concepts71 papers1 proposal

Generative models like diffusion and flow matching are typically understood through the lens of probability and statistics. We think of them as reversing a noising process or learning a vector field that transports a simple prior distribution to a complex data distribution.

However, there is an alternative, deeply intuitive perspective that unifies generation, guidance, and fine-tuning: Stochastic Optimal Control (SOC). Instead of viewing generation as a passive probabilistic simulation, we can view it as an active steering problem. Imagine a particle moving through a noisy fluid; our goal is to apply a time-dependent force—a control policy—that guides this particle from a simple starting state to a desired target distribution while expending the minimum amount of energy.

By reframing generative modeling as a control problem, we gain access to a rich mathematical toolkit that explains why these models work so well and provides principled algorithms for manipulating their outputs.

The Mathematical Duality: Scores and Value Functions

The foundational link between generative modeling and control theory rests on a striking mathematical duality. In stochastic optimal control, a system's trajectory is governed by a drift term (the control policy) and random noise. To find the best control policy, engineers use a Value Function $V(t, x)$, which represents the "remaining cost-to-go" or the expected future penalty from a given state $x$ at time $t$. The optimal control policy $\mathbf{u}^*(t, x)$ is simply the gradient of this value function:

$$\mathbf{u}^*(t, x) = \nabla V(t, x)$$

The evolution of this value function over time is governed by a fundamental partial differential equation known as the Hamilton-Jacobi-Bellman (HJB) equation.

As shown by Julius Berner et al. in An optimal control perspective on diffusion-based generative modeling, the log-density of a diffusion model can be mapped directly to the HJB equation via the Hopf–Cole transformation. This reveals that the score function $\nabla \log p_t(x)$—the very quantity that diffusion models are trained to estimate—is not just a statistical property. It is exactly the optimal control policy that steers a noise-driven process toward the target data distribution with minimal kinetic energy.

This framing was rigorously established in early work by Belinda Tzen et al. in Theoretical guarantees for sampling and inference in generative models with latent diffusions, proving that neural networks can efficiently steer simple diffusions to match complex targets by minimizing an energy cost.

Recognizing this duality allows researchers to design better training objectives. For instance, Carles Domingo-Enrich et al. introduced Stochastic Optimal Control Matching (SOCM), demonstrating that complex control problems can be represented as the gradient of a conditional expectation. This allows control problems to be trained using the stable, least-squares objectives originally developed for score matching, dramatically reducing gradient variance.

Guidance and Conditioning as Trajectory Steering

In practical applications, we rarely want to sample randomly from a model; we want to guide the generation based on a prompt, a condition, or a set of constraints. Standard diffusion models achieve this using heuristic "guidance weights" to balance the unconditional score and a classifier gradient, which often requires tedious manual tuning.

Under the control framework, conditioning is no longer a heuristic hack; it is a structured modification of the control objective. When we condition a model, we introduce a terminal reward or constraint that the particle must satisfy at the end of its trajectory.

Mathematically, this corresponds to Doob’s $h$-transform, a classical tool for conditioning stochastic processes. As explained by Alexander Denker et al. in DEFT: Efficient Fine-tuning of Diffusion Models by Learning the Generalised $h$-transform, the $h$-transform provides the exact mathematical force required to steer a random process toward a target conditional state. By training a small auxiliary network to learn this transform, they achieve efficient conditioning without altering the base model.

Standard Diffusion:  [Noise] --------(Score Field)--------> [Data Distribution]
                                          |
                                   (Heuristic Guidance)
                                          v
Guided Control:      [Noise] ----(Base + Value Gradient)----> [Target Condition]

Instead of relying on fixed guidance scales, we can use control theory to compute the mathematically optimal guidance strength dynamically. Iskander Azangulov et al. leveraged this in Adaptive Diffusion Guidance via Stochastic Optimal Control, solving the HJB equation to dynamically adapt the guidance force at each time step based on the specific sample being generated. Similarly, the authors of Variational Control for Guidance in Diffusion Models developed Diffusion Trajectory Matching (DTM), which optimizes step-by-step control signals to satisfy complex, non-linear constraints on the fly without requiring any model retraining.

Solving Inverse Problems and Model Alignment

The control perspective shines brightest when applied to downstream tasks like solving inverse problems (e.g., deblurring, super-resolution) or aligning models with human preferences.

Robust Inverse Problems via iLQR

When solving an inverse problem, we want to find a generative trajectory that terminates at a data point matching our observed corrupted measurements. Traditional methods treat this as a posterior sampling problem, which is highly sensitive to discretization errors—a single misstep in the early stages of generation can cause the trajectory to diverge entirely.

In Solving Inverse Problems via Diffusion Optimal Control, Henry Li et al. reframed this task as a discrete optimal control problem. Because the reverse diffusion process is a deterministic dynamical system when its noise is fixed, they applied the Iterative Linear Quadratic Regulator (iLQR)—a classic trajectory optimization algorithm from robotics. Instead of passively sampling, iLQR iteratively optimizes the control inputs over the entire trajectory to minimize the distance between the generated output and the measurements, making the reconstruction exceptionally robust to noise and non-linearities. A similar deterministic control approach is taken by Xingchao Liu et al. in FlowGrad: Controlling the Output of Generative ODEs with Gradients, optimizing control variables over ODE trajectories to achieve precise image manipulation.

Preference Alignment without Mode Collapse

Fine-tuning generative models to maximize a reward function (such as human aesthetic preferences) often leads to "mode collapse," where the model finds a single high-reward output and forgets the rest of the data distribution.

Control theory solves this by framing alignment as a control problem with a running cost that penalizes deviations from the original model's velocity field. As demonstrated by Zhen Liu et al. in Value Gradient Guidance for Flow Matching Alignment, matching the model's velocity field to the gradient of a learned value function allows for rapid alignment with human preferences while preserving the structural diversity of the original distribution. To scale this to massive models, Cristian Perez Jensen developed Value Matching: Scalable and Gradient-Free Reward-Guided Flow Adaptation, training a separate network to predict expected future rewards to guide the generation process without modifying the multi-billion parameter base model's weights.

Bridges Between Arbitrary Distributions: The Schrödinger Bridge

Standard diffusion models are fundamentally asymmetric: they always transport a simple, mathematically tractable prior (like a Gaussian) to a complex data distribution. But what if we want to transport directly between two arbitrary, complex distributions—such as translating an MRI scan to a CT scan, or simulating a molecule transitioning between two states?

This is the Schrödinger Bridge Problem (SBP), which seeks the most likely stochastic path that interpolates between two given probability distributions in finite time. SBP is the entropic generalization of optimal transport, and it is fundamentally an optimal control problem.

Standard Diffusion:  [Gaussian Noise] ------------------------> [Data Distribution]
Schrödinger Bridge:  [Data Distribution A] --(Optimal Control)--> [Data Distribution B]

By framing diffusion bridges through the lens of stochastic optimal control, researchers can unify disparate bridge architectures. Kaizhen Zhu et al. demonstrated this in UniDB: A Unified Diffusion Bridge Framework via Stochastic Optimal Control, proving that many existing diffusion bridge formulations are simply restricted cases of a broader SOC framework that differs only in how they penalize steering effort versus terminal accuracy.

This bridge formulation extends far beyond image generation. In molecular dynamics, finding rare transition paths (such as protein folding) is incredibly challenging because standard simulations get stuck in local energy minima. Lars Holdijk et al. solved this in PIPS: Path Integral Stochastic Optimal Control for Path Sampling in Molecular Dynamics by mapping molecular transition path sampling directly to the Schrödinger Bridge Problem. By using a neural network as an optimal controller, the model learns a bias potential that actively guides the molecular geometry across high-energy barriers, eliminating the need for hand-crafted, low-dimensional descriptors.

Whether generating an image, solving an inverse problem, or folding a protein, the stochastic optimal control perspective transforms generative modeling from a problem of passive estimation into one of active, optimal navigation.

Key Foundational Work

Theoretical guarantees for sampling and inference in generative models with latent diffusions (2019) — contributed 2 concept elements to this direction.
History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 73 papers
  2. Version 1 · 2026-10-04 · 110 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_stochastic_optimal_control_sampling_af5429a8,
  title = {stochastic optimal control sampling},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/stochastic-optimal-control-sampling/rdr_af5429a840794ea493daaf36a5cf4f70},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 2, frozen text):

@misc{lacuna_stochastic_optimal_control_sampling_af5429a8_v2,
  title = {stochastic optimal control sampling},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/stochastic-optimal-control-sampling/rdr_af5429a840794ea493daaf36a5cf4f70/v/2},
  note = {Version 2, frozen 2026-10-04. Accessed 2026-11-19},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake