Self-Tuning Markov Chain Monte Carlo via Online Adaptation

This direction is merged into another direction: adaptive proposal distribution. Its last version stays readable and citable here.

Version 1 of 193 papers112 conceptsfrozen 2026-10-04

Markov Chain Monte Carlo (MCMC) is the workhorse of Bayesian inference, but its efficiency traditionally depends on a "Goldilocks" problem. To explore a probability landscape, a sampler must choose a step size: too small, and it crawls through the space with high correlation; too large, and it constantly overshoots, leading to high rejection rates. Historically, researchers spent hours "pilot-tuning" these parameters.

The research direction across these papers describes a shift toward Adaptive MCMC, where the algorithm learns the geometry of the target distribution on the fly. By treating the sampler's own history as data, these methods transform MCMC from a fragile manual process into a robust, "black-box" tool.

The Robbins-Monro Feedback Loop

The most fundamental form of adaptation targets the acceptance rate. Theoretical work suggests that for many high-dimensional problems, the "optimal" acceptance rate for a random-walk sampler is approximately 23.4%. Paul H. Garthwaite and Yanan Fan in Adaptive Optimal Scaling of Metropolis-Hastings Algorithms Using the Robbins-Monro Process demonstrate how to automate this using a stochastic approximation known as the Robbins-Monro process.

The intuition is simple: if the last move was accepted, slightly increase the step size; if it was rejected, decrease it. By scaling these adjustments so they diminish over time, the sampler eventually settles into the "Goldilocks zone" without breaking the mathematical requirements for convergence (ergodicity). This mechanism is now a staple in modern frameworks like AdvancedHMC.jl, as described by Hong Ge and colleagues, where it is used to tune the integration step size during an initial "warm-up" phase.

Learning the Landscape via Covariance

Step size is only half the battle. Most real-world distributions are not spherical; they are stretched into long ridges or tilted by correlations between variables. A sampler that only moves in circles will struggle to navigate a narrow, diagonal valley.

Adaptive algorithms solve this by estimating the empirical covariance of the target distribution. Jinhong Yuan, Gareth W. Peters, and Ido Nevat show in Channel Tracking for Relay Networks via Adaptive Particle MCMC how a mixture-of-Gaussians proposal can dynamically adjust its orientation. By learning which directions have high variance, the sampler stretches its "search ellipse" to match the target's shape.

In more complex settings, this is framed as a filtering problem. Isambi Sailon Mbalawata and Heikki Haario in Adaptive Metropolis Algorithm Using Variational Bayesian Adaptive Kalman Filter treat the optimal proposal covariance as a hidden state, using a Kalman filter to track it as new samples arrive. This allows the sampler to adapt even when the local geometry changes across different regions of the state space.

Tuning-Free Sampling with Slice Methods

While some algorithms adapt by looking backward at their history, others adapt "locally" within a single step. Slice sampling is a prominent example of this. Instead of a fixed proposal distribution, it defines a "slice" of the density and draws a sample uniformly from it.

Iain Murray 0001 argues in Slice sampling covariance hyperparameters of latent Gaussian models that this approach is inherently robust because it automatically adjusts its scale based on the local density. If the slice is too wide, a "shrinkage" procedure narrows it until a valid point is found. This makes it particularly effective for "funnel-shaped" distributions where the scale varies wildly. Philip Schär and Daniel Rudolf further refined this with Gibbsian Polar Slice Sampling, which decomposes high-dimensional moves into radial and directional updates that require zero manual tuning.

Population-Based and Non-Reversible Adaptation

A newer frontier involves using multiple "particles" to learn the distribution collectively. In Sample Adaptive MCMC, Michael Zhu proposes using a population of $N$ points to calculate an empirical mean and covariance. Unlike traditional adaptive MCMC, which can sometimes "forget" the Markov property and fail to converge, population-based methods can remain valid Markov chains while benefiting from the collective intelligence of the particle cloud.

Similarly, in Sequential Monte Carlo (SMC) frameworks, Christian Schäfer demonstrates in Sequential Monte Carlo on large binary sampling spaces how the distribution of the entire particle set can be used to calibrate proposal distributions for the next generation.

Finally, the field is moving toward non-reversible samplers, such as the Stochastic Bouncy Particle Sampler by Ari Pakman and Liam Paninski. These algorithms use "velocity" and "bounces" rather than random walks. By adaptively preconditioning the velocity based on the target's covariance, these samplers can glide through ill-conditioned spaces that would trap traditional Metropolis-Hastings algorithms.

Why It Works: The Intuition of Diminishing Adaptation

The core insight across all these papers is that information is not free, but it is reusable. Every rejected sample in a standard MCMC run is "wasted" computation. Adaptive MCMC captures that waste, using the failures to learn where the boundaries of the distribution lie and using the successes to learn the direction of the ridges.

The mathematical "magic" that makes this valid is diminishing adaptation: as the sampler gains more information, it makes smaller and smaller changes to its strategy. This ensures that while the algorithm is learning early on, it eventually stabilizes into a standard Markov chain that converges to the true posterior, providing the efficiency of a hand-tuned sampler with the ease of an automated one.

Papers in this version

Showing 60 of 93 papers.

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_self_tuning_markov_chain_monte_carlo_via_33de4ee0,
  title = {Self-Tuning Markov Chain Monte Carlo via Online Adaptation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/self-tuning-markov-chain-monte-carlo-via-online-adaptation/rdr_33de4ee0e13949ebae6372bd952ad5f6},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_self_tuning_markov_chain_monte_carlo_via_33de4ee0_v1,
  title = {Self-Tuning Markov Chain Monte Carlo via Online Adaptation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/self-tuning-markov-chain-monte-carlo-via-online-adaptation/rdr_33de4ee0e13949ebae6372bd952ad5f6/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}