Variational Hamiltonian Monte Carlo via Score Matching

Cheng Zhang, Babak Shahbaba, Hongkai Zhao

Abstract

Traditionally, computational Bayesian statistics has been divided into two primary paradigms: variational methods and Markov chain Monte Carlo (MCMC). While MCMC offers asymptotic exactness, it often struggles with computational scaling; conversely, variational methods provide speed at the cost of restrictive distributional assumptions. In this paper, we introduce Variational Hamiltonian Monte Carlo (VHMC), a hybrid approach that incorporates variational approximation into the Hamiltonian Monte Carlo (HMC) framework. By utilizing a fast and flexible surrogate function—constructed as an optimized additive model of random bases—we reduce the expensive gradient computations typically required by HMC. We demonstrate that this "free-form" approximation, trained via score matching, allows for efficient sampling in high-dimensional and large-scale data regimes, outperforming traditional variational and stochastic gradient methods in both accuracy and convergence speed.

Introduction: The Big Data Bottleneck in HMC

In Bayesian inference, the goal is often to sample from a complex posterior distribution. Hamiltonian Monte Carlo (HMC) has emerged as a gold standard for this task because it leverages gradient information to explore the parameter space more efficiently than random-walk Metropolis-Hastings. However, HMC faces a significant challenge in the era of big data: every step of the simulation requires calculating the gradient of the log-posterior across the entire dataset.

Existing attempts to scale HMC, such as Stochastic Gradient MCMC (SGLD), utilize data subsampling to reduce this burden. However, the resulting gradient noise can destabilize the Hamiltonian dynamics, necessitating smaller step sizes and slower convergence. On the other end of the spectrum, Variational Bayes (VB) is computationally efficient but often relies on rigid parametric families (like Gaussians) that fail to capture the true complexity of the posterior. VHMC seeks to bridge this gap by using a neural network surrogate to approximate the gradient landscape, providing the speed of variational methods with the flexibility of MCMC.

The VHMC Framework

1. Neural Network Surrogates and Extreme Learning

To avoid the cost of querying the true potential energy function $U(\theta)$ at every leapfrog step, VHMC employs a Single-Hidden Layer Feedforward Network (SLFN) as a surrogate. To maintain high computational efficiency, the authors utilize an "Extreme Learning Machine" (ELM) architecture. In this setup, the weights of the hidden layer are randomly assigned and kept fixed. Consequently, training the surrogate reduces to a simple linear regression problem for the output weights, allowing for extremely fast updates as the MCMC chain progresses.

2. Training via Score Matching

Because HMC dynamics are driven by the gradient of the log-density (the "score") rather than the density value itself, the surrogate is trained using score matching. Instead of minimizing the difference between the surrogate and the true function values, the algorithm minimizes the squared distance between their gradients. This ensures that the surrogate accurately reflects the "forces" that guide the sampler through the parameter space, maintaining stable Hamiltonian trajectories even if the absolute density values are slightly shifted.

3. Free-Form Variational Inference

Unlike standard VB, which forces the posterior into a predefined shape, the surrogate in VHMC induces a "free-form" variational approximation. The sampler explores a landscape defined by the neural network, which can adapt to multi-modal or highly non-Gaussian distributions. This allows VHMC to capture structural nuances that traditional variational methods would miss.

4. The Cold Start Strategy

At the start of the sampling process, there is insufficient information to train an accurate surrogate. VHMC addresses this using a smooth transition schedule. The algorithm begins with a simple Laplace approximation (a Gaussian fit at the mode) and gradually shifts its reliance toward the neural network surrogate as more samples are collected and the surrogate's accuracy improves.

Experimental Results

The authors evaluated VHMC across several benchmarks, ranging from synthetic tests to high-dimensional real-world datasets:

  • Accuracy: In Probit regression experiments ($N=10,000$), VHMC achieved a lower Root Mean Squared Error (RMSE) than standard Variational Bayesian Expectation Maximization (VBEM). This confirms that the free-form surrogate is more capable of representing the true posterior than rigid Gaussian approximations.
  • Convergence Speed: On Magnetoencephalography (MEG) data ($N=17,730$) used for Independent Component Analysis (ICA), VHMC converged to the true posterior significantly faster than SGLD. While SGLD is fast per iteration, its inherent noise requires a much smaller step size. VHMC, by contrast, takes large, efficient HMC steps on a smooth surrogate landscape.
  • Efficiency: VHMC maintained high acceptance probabilities (typically 70-85%) while drastically reducing the number of full-likelihood evaluations required, demonstrating its viability for large-scale Bayesian problems.

Conclusion and Future Directions

Variational Hamiltonian Monte Carlo offers a promising path for scalable Bayesian inference. By focusing on gradient matching and utilizing fast-training neural surrogates, it preserves the strengths of HMC while mitigating its primary computational weakness.

Future work may explore how VHMC scales with even higher dimensionality. As the parameter count increases, the number of random basis functions required for the surrogate may grow. The authors suggest that exploiting graphical model structures or conditional independence could lead to more structured, efficient surrogates, further extending the reach of VHMC in complex, high-dimensional applications.

Summary updated

Abstract

Traditionally, the field of computational Bayesian statistics has been divided into two main subfields: variational methods and Markov chain Monte Carlo (MCMC). In recent years, however, several methods have been proposed based on combining variational Bayesian inference and MCMC simulation in order to improve their overall accuracy and computational efficiency. This marriage of fast evaluation and flexible approximation provides a promising means of designing scalable Bayesian inference methods. In this paper, we explore the possibility of incorporating variational approximation into a state-of-the-art MCMC method, Hamiltonian Monte Carlo (HMC), to reduce the required expensive computation involved in the sampling procedure, which is the bottleneck for many applications of HMC in big data problems. To this end, we exploit the regularity in parameter space to construct a free-form approximation of the target distribution by a fast and flexible surrogate function using an optimized additive model of proper random basis, which can also be viewed as a single-hidden layer feedforward neural network. The surrogate function provides sufficiently accurate approximation while allowing for fast computation in the sampling procedure, resulting in an efficient approximate Bayesian inference algorithm. We demonstrate the advantages of our proposed method using both synthetic and real data problems.
BibTeX
@inproceedings{zhang2017variational,
  author = {Cheng Zhang and Babak Shahbaba and Hongkai Zhao},
  title = {{Variational Hamiltonian Monte Carlo via Score Matching}},
  booktitle = {Bayesian},
  year = {2017},
  doi = {10.1214/17-ba1060},
}
Figure