Amortized Initialization for Markov Chain Monte Carlo

68 concepts47 papers1 proposal

Markov Chain Monte Carlo (MCMC) and Langevin dynamics are incredibly powerful tools for exploring complex, high-dimensional probability distributions. However, they suffer from a notorious computational bottleneck: the burn-in phase.

When an MCMC chain is initialized from random noise or an uninformed prior, it must spend significant computational time traversing low-probability regions before it reaches the target distribution's "typical set." In multimodal landscapes, this problem worsens exponentially; chains can easily become trapped in local modes, requiring an impractical number of iterations to cross high-energy barriers.

The core insight behind amortized and warm-start initialization is simple: Instead of forcing a sampler to discover the structure of a distribution from scratch, we can use a fast approximation to drop the chain directly onto or near the high-probability manifold. This shifts the role of MCMC from global structure discovery to local uncertainty refinement, drastically reducing the number of steps required for convergence.


Amortized Samplers and Cooperative Learning

One of the most robust ways to generate a high-quality starting point is to train a neural network to predict the posterior distribution. This network acts as an "amortized sampler"—it incurs an upfront training cost but can instantly predict a warm start for any new data point during inference.

[Random Noise] ───(Slow Burn-In)───► [Target Manifold] ───(Local Exploration)───► [Samples]
                                           ▲
[Data Point] ───► [Amortized Encoder] ─────┘ (Instant Jump)

Xie et al. demonstrate this synergy by pairing an Energy-Based Model (EBM) with a Variational Auto-Encoder (VAE). The VAE serves as an amortized sampler that maps structured latent representations directly to the target distribution. By starting from this informed initialization rather than random noise, the EBM achieves high-quality synthesis with as few as 15 to 50 Langevin steps, bypassing local mode entrapment. Similarly, Sun et al. utilize an inference network to initialize Langevin dynamics for unsupervised representation learning, showing that a single refinement step from an informed starting point can outperform dozens of iterations initiated from noise.

This relationship can be optimized through cooperative learning, where the initializer and the refiner are trained jointly. In the "CoopFlow" framework proposed by Xie et al., a Normalizing Flow is trained to jump-start a short-run Langevin MCMC chain. As training progresses, the initializer learns to handle the macroscopic structure of the data, allowing the MCMC refiner to focus entirely on fine-grained details. Zahid et al. apply a similar principle to predictive coding, using an auxiliary encoder network to place the initial state of a Langevin chain into high-probability latent regions, accelerating convergence and reducing training iterations.


Exploiting Temporal and Sequential Continuity

In sequential settings—such as online learning, reinforcement learning, or iterative model training—the target distribution changes incrementally over time. Re-initializing an MCMC sampler from scratch at every single time step discards valuable geometric information. Instead, we can use the final state of the previous step as a warm start for the current step.

This strategy is highly effective for scaling Thompson Sampling to high-dimensional contextual bandits. Mazumdar et al. and Xu et al. demonstrate that initializing a Langevin MCMC sampler with the parameter state from the preceding decision round allows the chain to track the shifting posterior with minimal computational overhead. Because the posterior updates incrementally as new data arrives, the previous state is already near stationarity, making real-time iterative sampling computationally viable.

Time t-1:  [Posterior t-1] ───► Final State ──┐ (Warm Start)
                                               ▼
Time t:    [Posterior t]   ◄───────────────────┘ (Rapid Convergence)

Beyond bandits, this temporal warm-start mechanism stabilizes training loops. Huix et al. show that initializing MCMC chains at the mode of the posterior (found via preliminary gradient descent steps) provides tight regret and complexity bounds. In generative zero-shot learning, Xie et al. accelerate latent variable inference by initializing the search in each training iteration with the latent factors discovered in the previous step. In reinforcement learning, Lei et al. initialize action-sampling MCMC chains using the current predictions of a generator policy network, achieving the real-time inference speeds necessary for robotic control.


Deterministic Search and Coarse Approximations

When a learned neural initializer is unavailable, fast deterministic algorithms or coarse mathematical approximations can provide an excellent baseline initialization.

For example, non-parametric Bayesian models like Dirichlet process mixture models are notoriously slow to sample. However, Hal Daumé III demonstrates that if you use deterministic search algorithms (like A* or beam search) to find a Maximum A Posteriori (MAP) estimate first, you can use that point estimate to initialize a Gibbs sampler. The sampler is spared from struggling with the initial discovery of cluster structures and can immediately focus on exploring local uncertainty. Rai et al. apply this exact logic to the Indian Buffet Process, combining the speed of beam search with the uncertainty quantification of MCMC.

Alternatively, we can compute a cheap, coarse discretization of the target distribution to guide the sampler:

  • Discretized Grids: De la Fuente et al. introduce Discretized Approximate Ancestral Sampling, showing that approximate samples generated via interpolation on a discrete grid can warm-start MCMC chains. This prevents the divergence common to randomly initialized chains and yields high-precision samples in very few steps.
  • Sublinear Seeding: The authors of Approximate K-Means++ in Sublinear Time replace exact, expensive distance distributions with a local MCMC sampler that approximates the target distribution, finding high-quality initial cluster centroids in sublinear time.
  • Deterministic Pre-fitting: In medical imaging, Hernández et al. integrate fast deterministic fits as a precursor to Bayesian sampling for diffusion MRI, accelerating the convergence of stochastic chains across thousands of voxels.

Manifold-Proximal and Data-Based Initialization

For generative tasks like image synthesis and inverse problem solving, the target distribution lives on a low-dimensional manifold embedded within a massive feature space. Initializing a sampler far from this manifold causes it to get lost in empty space.

To circumvent this, samplers can be anchored using real data points or pre-trained generative priors. Turner et al. introduce Metropolis-Hastings GANs, which initialize MCMC chains using real data points to eliminate the burn-in period entirely, ensuring that subsequent sampling steps remain in high-probability regions of the feature space. For training energy-based models, Berglund et al. show that initializing the negative phase chains at current data points reduces gradient variance by allowing noise cancellation between the positive and negative samples. Similarly, Kang et al. initialize MCMC sampling during GAN compression with the current output of a student network to mitigate the computational overhead of energy-based training.

Standard Diffusion:  [Pure Noise] ───────────────────────────────► [Data Manifold] (Long Path)
Warm-Start:          [MCMC Intermediate Noise] ──► [Data Manifold] (Short Path)

This concept is particularly powerful for accelerating diffusion models. Kim et al. demonstrate that initializing the reverse diffusion process from intermediate noise levels discovered via MCMC—rather than starting from pure Gaussian noise—drastically shortens the numerical integration path. This warm-start approach minimizes the accumulation of discretization errors and allows numerical solvers to reach high-quality results with significantly fewer function evaluations. For complex inverse problems like image deblurring, Xun et al. use a pre-trained generative model to provide a warm start for posterior sampling, placing the initial state directly on the data manifold where score estimates are most reliable.

Finally, this perspective yields profound theoretical guarantees for multimodal sampling. Gay et al. prove that "data-based initialization"—starting a Markov chain with a small set of samples drawn from the target distribution—bypasses the exponential time typically required to transition between isolated modes. Because the chain is already distributed across the correct modes, it only needs to mix locally within each component, turning an otherwise intractable global sampling problem into a highly efficient local one.

Key Foundational Work

Beam Search based MAP Estimates for the Indian Buffet Process (2011) — contributed 1 concept elements to this direction.
Fast search for Dirichlet process mixture models (2007) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_amortized_mcmc_initialization_e7225a1d,
  title = {amortized MCMC initialization},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/amortized-mcmc-initialization/rdr_e7225a1d705a4f1b80afafe8a7bc9cf6},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_amortized_mcmc_initialization_e7225a1d_v1,
  title = {amortized MCMC initialization},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/amortized-mcmc-initialization/rdr_e7225a1d705a4f1b80afafe8a7bc9cf6/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake