Sampling from unnormalized probability densities—where we know the energy function but not the normalizing constant—is a fundamental challenge in Bayesian inference and statistical mechanics. While diffusion models have revolutionized generative modeling, they typically rely on large datasets. When we instead try to "solve" a distribution defined by an energy function, standard variational objectives like the Kullback-Leibler (KL) divergence often fail, leading to mode collapse where the sampler ignores significant portions of the distribution.
Recent work by Julius Berner, Lorenz Richter, and others has established a more robust framework for this problem by treating sampling as a stochastic optimal control problem and introducing the log-variance divergence as a superior optimization objective.
The Control Perspective on Diffusion
The core insight, detailed in An optimal control perspective on diffusion-based generative modeling, is that the process of transforming a simple base distribution (like a Gaussian) into a complex target can be viewed through the lens of the Hamilton–Jacobi–Bellman (HJB) equation. In this view, the "score" or "drift" of the diffusion process is an optimal control that minimizes the energy required to reach the target distribution.
In standard variational inference, we minimize the KL divergence between the path measure of our controlled diffusion and the target path measure. However, the KL objective is "mode-seeking"; if the target has multiple peaks separated by low-density valleys, the model often collapses into a single peak to minimize the penalty, losing the global structure of the distribution.
Why Log-Variance Divergence Works
To solve this, Richter and Nüsken proposed the log-variance loss in Solving high-dimensional Hamilton-Jacobi-Bellman PDEs using neural networks. Instead of minimizing the average log-ratio of densities (the KL approach), this objective minimizes the variance of the log-ratio between the path measures.
Intuition for the log-variance loss:
- Zero Variance at Optimum: When the model perfectly matches the target, the ratio of the path measures becomes a constant (the normalizing constant). At this point, the variance of the log-ratio is exactly zero.
- Global Coverage: Because the loss penalizes any deviation from a constant ratio across the entire path space, it forces the model to account for all regions of the target distribution simultaneously. This prevents the sampler from "ignoring" modes, as doing so would create high variance in the log-ratio.
- Gradient Stability: As shown in Improved sampling via learned diffusions, this objective eliminates the need to differentiate through the SDE solver, leading to more stable training and vanishing gradient variance as the model converges.
Applications in Rare Events and Multi-modality
This framework is particularly powerful for scientific domains where the "ground truth" is an energy landscape rather than a dataset.
- Molecular Dynamics: In Transition Path Sampling with Improved Off-Policy Training of Diffusion Path Samplers, Kiyoung Seong and colleagues use log-variance divergence to sample rare transition pathways, such as protein folding. The objective allows the model to discover multiple distinct pathways between stable states without requiring human-defined "collective variables."
- Multi-modal Sampling: Maxence Noble and Alain Durmus demonstrate in Learned Reference-based Diffusion Sampler for multi-modal distributions that this approach accurately recovers the relative weights of different modes, a task where standard MCMC and KL-based diffusion samplers frequently fail.
By shifting the objective from "matching the average" to "minimizing the variance," these researchers have turned diffusion models into a robust tool for exploring complex, high-dimensional energy landscapes.