Abstract
Traditionally, the field of computational Bayesian statistics has been divided into two main subfields: variational methods and Markov chain Monte Carlo (MCMC). In recent years, however, several methods have been proposed based on combining variational Bayesian inference and MCMC simulation in order to improve their overall accuracy and computational efficiency. This marriage of fast evaluation and flexible approximation provides a promising means of designing scalable Bayesian inference methods. In this paper, we explore the possibility of incorporating variational approximation into a state-of-the-art MCMC method, Hamiltonian Monte Carlo (HMC), to reduce the required gradient computation in the simulation of Hamiltonian flow, which is the bottleneck for many applications of HMC in big data problems. To this end, we use a free-form approximation induced by a fast and flexible surrogate function based on single-hidden layer feedforward neural networks. The surrogate provides sufficiently accurate approximation while allowing for fast exploration of parameter space, resulting in an efficient approximate inference algorithm. We demonstrate the advantages of our method on both synthetic and real data problems.
Introduction
In Bayesian statistics, practitioners often face a trade-off between the high accuracy of Markov Chain Monte Carlo (MCMC) and the computational efficiency of Variational Bayes (VB). While MCMC methods like Hamiltonian Monte Carlo (HMC) are considered the gold standard for generating unbiased samples from complex posterior distributions, their reliance on full-dataset gradient evaluations makes them prohibitively expensive for large-scale datasets. Conversely, VB methods offer speed by approximating the posterior with a simpler distribution, but they often introduce significant bias due to their rigid parametric forms.
This paper introduces Variational Hamiltonian Monte Carlo (VHMC), a framework that bridges this gap. By utilizing neural network surrogates trained via score matching, VHMC retains the efficient state-space exploration of HMC while drastically reducing the computational cost associated with big data.
The Computational Bottleneck of HMC
HMC leverages the geometry of the target distribution by simulating Hamiltonian dynamics. This process requires calculating the gradient of the log-posterior (the "score"), which involves iterating over every observation in the dataset. In the era of big data, performing these calculations dozens of times per sample is a major bottleneck. While stochastic gradient versions of HMC exist, they introduce noise that can degrade the algorithm's performance, often requiring a meticulously tuned step-size schedule to ensure convergence. VHMC addresses this by replacing the expensive true gradient with a fast, smooth surrogate model.
The VHMC Framework
The core of the VHMC approach lies in its use of a "free-form" variational approximation. Unlike traditional VB, which might force a posterior to fit a Gaussian shape, VHMC uses a flexible neural network to learn the distribution's structure.
1. Neural Network Surrogates
The algorithm employs a Single-Hidden Layer Feedforward Network (SLFN) to approximate the potential energy function (the negative log-posterior). Because neural networks are universal approximators, this allows the algorithm to capture complex, multi-modal, or skewed distributions that fixed-form variational methods would miss.
2. Efficient Training via Extreme Learning Machines (ELM)
To ensure the surrogate itself does not become a computational burden, the authors utilize the Extreme Learning Machine (ELM) framework. In an ELM, the internal weights of the hidden layer are assigned randomly and kept fixed. Only the output weights are trained, transforming a non-linear optimization problem into a simple, high-speed linear regression task.
3. Score Matching
Since HMC only requires the gradient of the log-posterior to simulate movement, the surrogate is trained using score matching. This objective function minimizes the difference between the gradient of the surrogate and the gradient of the true log-posterior. By focusing on the "score" rather than the probability density values, the model avoids the need to calculate the intractable normalizing constant of the posterior.
4. Regularization and the Cold Start Problem
To stabilize the surrogate during the early stages of sampling—when data points are sparse—the authors incorporate a Laplace approximation as a regularizer. This provides a sensible initial guess for the distribution's shape, which is gradually phased out as the neural network gathers more information and becomes the primary driver of the Hamiltonian simulation.
Experimental Evaluation
The authors validated VHMC across several diverse tasks, comparing it against standard Variational Bayesian Expectation Maximization (VBEM), standard HMC, and Stochastic Gradient Langevin Dynamics (SGLD).
- Synthetic Probit Regression: In a controlled environment ($N=10,000, d=5$), VHMC achieved a lower Root Mean Squared Error (RMSE) than standard VBEM, demonstrating that its free-form nature allows for a more accurate fit than traditional parametric approximations.
- UCI 'adult' Dataset: On a real-world logistic regression task ($N=32,561$), VHMC demonstrated superior exploration efficiency. By measuring the Effective Sample Size (ESS) per second, the researchers showed that VHMC provides more independent samples in less time than standard HMC by avoiding full-dataset scans.
- MEG Dataset (ICA): In a high-dimensional Independent Component Analysis problem, VHMC converged to the true posterior significantly faster than SGLD. While SGLD iterations are fast, the noise inherent in stochastic gradients requires a diminishing step size that slows convergence; VHMC’s smooth surrogate allows for consistently large, purposeful steps.
Key Insights
VHMC effectively performs a form of "implicit subsampling." By training a surrogate, the algorithm learns the global regularity of the data. Once the surrogate is trained, the sampler can navigate the parameter space without repeatedly revisiting every individual data point. This makes VHMC particularly suited for problems where the data is redundant or follows a consistent global structure.
Conclusion and Future Work
Variational Hamiltonian Monte Carlo represents a significant step toward scalable Bayesian inference. By combining the flexibility of neural networks with the rigorous sampling of HMC, it offers a path forward for complex models that were previously too computationally expensive to evaluate.
Future research may focus on automating the selection of hyperparameters, such as the number of hidden neurons and the transition schedule for the Laplace regularizer. Additionally, exploring more complex surrogate architectures, such as deep networks or graphical models, could further enhance the method's ability to scale to extremely high-dimensional parameter spaces.
Summary updated