Generative Modeling from Unnormalized Energy Functions

33 concepts25 papers

Standard generative models—such as Generative Adversarial Networks (GANs), Normalizing Flows, and Diffusion Models—are traditionally data-hungry. They learn a probability distribution $p(x)$ by optimizing parameters over thousands or millions of empirical samples.

But in many scientific domains, we face the exact inverse scenario: we do not have a dataset of samples, but we do possess the exact mathematical formula for the system's unnormalized density. In physics, chemistry, and Bayesian statistics, this is typically expressed as an energy function $E(x)$, where the true probability density is given by the Boltzmann distribution:

$$p(x) = \frac{\exp(-E(x))}{Z}$$

The fundamental bottleneck is the normalizing constant (or partition function) $Z = \int \exp(-E(x)) dx$, which is highly intractable in high dimensions.

To bypass this bottleneck, a rapidly growing research direction focuses on generative modeling and sampling directly from unnormalized energy functions. By leveraging the properties of the score function (the gradient of the log-density), researchers are developing methods to either sample from these distributions training-free, or to train neural samplers without a single piece of empirical data.

The Core Insight: The Score Function Cancels the Partition Function

Why can we model a distribution without knowing its normalizing constant $Z$? The answer lies in taking the spatial gradient of the log-density, known as the score function:

$$\nabla_x \log p(x) = \nabla_x \left( -E(x) - \log Z \right) = -\nabla_x E(x)$$

Because $Z$ does not depend on $x$, its derivative vanishes. This means that even if $Z$ is completely unknown, we have perfect, exact access to the score function of the target distribution simply by taking the gradient of our energy function.

This mathematical convenience forms the bedrock of two distinct paradigms: on-the-fly score estimation (which samples without neural network training) and amortized data-free training (which distills the energy landscape into a fast neural sampler).

Traditional Generative Modeling:
[Empirical Samples] ───(Score Matching / ML)───► [Learned Score Network] ───► [Samples]

Energy-Driven Generative Modeling:
[Energy Function E(x)] ───(Direct Gradient)───► [Exact Score -∇E(x)] ───► [Samples / Amortized Model]

On-the-Fly Score Estimation (Training-Free Sampling)

If we can compute $-\nabla_x E(x)$ at any point, we can theoretically use Langevin dynamics to navigate the probability landscape. However, standard Langevin dynamics struggle with multi-modal distributions and high-dimensional spaces. To solve this without the overhead of training a global neural network, recent methods estimate or correct the score locally on the fly.

Multi-Scale SDEs and Stochastic Averaging

Instead of relying on a static gradient, Cordero Encinar et al. introduce a "slow-fast" system of stochastic differential equations (SDEs). By running two coupled processes at different timescales, the authors demonstrate that the score can be estimated in real-time through stochastic averaging. This allows the sampler to adaptively smooth out local noise and navigate complex landscapes without any offline optimization or pre-trained networks.

Local Particle Approximations for Zeroth-Order Settings

What if the energy function is a black box where gradients ($\nabla_x E(x)$) are unavailable, and we can only query point evaluations $E(x)$? Takagi et al. address this by treating a cloud of co-evolving particles as a local Gaussian mixture. By querying the energy function at these particle locations, the local geometry of the surrogate distribution can be used to estimate the score on the fly using only zeroth-order queries.

Correcting the Score via Sequential Monte Carlo

When moving backwards through a diffusion process training-free, local score approximations inevitably accumulate errors. To guarantee statistical exactness, Wu et al. propose Reverse Diffusion Sequential Monte Carlo (RDSMC). At each discrete time step of the reverse diffusion process, the authors run an internal annealed importance sampling loop. This acts as a local correction mechanism, allowing the sampler to provide unbiased estimates of the normalization constant $Z$ alongside exact samples.

Amortized Inference (Data-Free Training)

While on-the-fly estimation avoids training, it requires expensive energy or gradient evaluations at every single sample step. Amortized inference aims to shift this computational burden to an offline training phase, training a neural network (like a normalizing flow or a diffusion model) to become a fast, direct sampler using only the energy function.

Variational Objectives on Energy Landscapes

The most direct way to train a model without data is to minimize the Kullback-Leibler (KL) divergence between the model's distribution $q_\theta(x)$ and the target distribution $p(x) \propto \exp(-E(x))$. For example, Shanahan et al. train gauge-equivariant normalizing flows by minimizing the KL divergence directly against a known physical action in quantum chromodynamics, effectively learning a "trivializing map" without empirical data. Similarly, Máté et al. frame the transformation of probability densities as a fluid dynamics problem, solving the continuity equation to train continuous normalizing flows purely from energy formulas.

The Exploration Problem and How to Fix It

Training a model purely on an energy function introduces a severe optimization challenge: mode collapse. If a neural network has never generated a sample near a specific energy minimum, it will never receive the gradient signal telling it that the mode exists.

To break this vicious cycle, researchers combine generative training with active exploration mechanisms:

  • Prioritized Replay Buffers: The Flow Annealed Importance Sampling Bootstrap (FAB) framework developed by Midgley et al. utilizes Annealed Importance Sampling (AIS) to actively hunt for hidden modes, storing discovered high-probability states in a replay buffer to continuously challenge and update the normalizing flow.
  • Off-Policy Exploration: Sendera et al. leverage off-policy GFlowNet objectives paired with Langevin-based local exploration, ensuring that the model explores the energy landscape broadly rather than sinking into the nearest local minimum.
  • Importance Weighting: To force a diffusion sampler to cover all modes equally, Wang et al. introduce Importance Weighted Score Matching (IWSM). By re-weighting the score-matching objective with self-normalized importance sampling, they align the training objective with a mode-covering forward KL divergence.

High-Accuracy Corrections and Transport Maps

Even when a neural sampler or an on-the-fly score estimator is imperfect, we can use the exact score information to perform rigorous statistical corrections.

Historically, high-accuracy correction techniques like rejection sampling required evaluating the full probability density function $p(x)$, which means knowing $Z$. Chen et al. shattered this limitation by introducing First-Order Rejection Sampling (FORS). The authors proved that evaluating a stochastic path integral of the score function along a diffusion trajectory is sufficient to compute an exact, unbiased acceptance probability. This allows for exact-style rejection sampling using only first-order gradient information.

[Prior / Base Distribution] 
       │
       ▼
[Stochastic Path Integral of Score] ──► Computes Unbiased Acceptance Probabilities
       │
       ▼
[First-Order Rejection Sampling] ──► Guarantees Exact Target Samples (No Z required)

Furthermore, this energy-driven perspective allows for structured transport between completely different modalities. Tamogashev et al. extended the Schrödinger bridge framework to "data-to-energy" transport. Their method learns to construct a stochastic path that bridges a data-rich source distribution (like an empirical image dataset) directly to a target distribution defined solely by an unnormalized energy landscape, preserving semantic structure without requiring target samples.

By shifting the foundational premise of generative modeling away from empirical datasets and toward mathematical energy functions, this paradigm bridges classical statistical mechanics with deep generative architectures, providing a faster, more scalable foundation for scientific computing.

Key Foundational Work

Proposal of a Score Based Approach to Sampling Using Monte Carlo Estimation of Score and Oracle Access to Target Density (2022) — contributed 2 concept elements to this direction.
Variational Annealing of GANs: A Langevin Perspective (2019) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_on_the_fly_score_estimation_ab3b8b5b,
  title = {on-the-fly score estimation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/on-the-fly-score-estimation/rdr_ab3b8b5b302a46ba9010c5013a082cbd},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 1, frozen text):

@misc{lacuna_on_the_fly_score_estimation_ab3b8b5b_v1,
  title = {on-the-fly score estimation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/on-the-fly-score-estimation/rdr_ab3b8b5b302a46ba9010c5013a082cbd/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}

See something wrong? Report a mistake