Abstract
Recent advances in stochastic gradient variational inference have made it possible to perform variational Bayesian inference with posterior approximations containing auxiliary random variables. This enables us to explore a new synthesis of variational inference and Monte Carlo methods where we incorporate one or more steps of MCMC into our variational approximation. By doing so we obtain a rich class of inference algorithms bridging the gap between variational methods and MCMC, and offering the best of both worlds: fast posterior approximation through the maximization of an explicit objective, with the option of trading off additional computation for additional accuracy. We describe the theoretical foundations that make this possible and show some promising first results.
Introduction
In Bayesian machine learning, calculating the posterior distribution $p(z|x)$—the updated belief about latent variables $z$ given observed data $x$—is often computationally intractable. Traditionally, researchers have relied on two distinct paradigms:
- Variational Inference (VI): Fast and optimization-based, but often biased due to rigid parametric assumptions (e.g., Gaussian approximations).
- Markov Chain Monte Carlo (MCMC): Asymptotically exact but slow, lacking a clear optimization objective for tuning parameters.
This paper introduces a hybrid approach that bridges this gap. By treating MCMC steps as "auxiliary variables" within a variational framework, the authors derive a differentiable objective that allows models to refine a simple variational approximation using MCMC dynamics while maintaining a formal objective to optimize.
Methodology: MCMC as a Variational Distribution
The core innovation lies in interpreting a sequence of MCMC steps as a complex variational distribution. By treating intermediate states of a Markov Chain as auxiliary random variables, the authors derive a new variational lower bound, $\mathcal{L}_{aux}$.
To ensure the objective is tractable, they introduce an "auxiliary inference distribution" (or inverse model) $r$ that predicts the previous state of the chain given the current one. This framework is specifically applied to Hamiltonian Variational Inference (HVI), which utilizes Hamiltonian Monte Carlo (HMC) dynamics. In HVI, the parameters of the HMC chain—such as step sizes and mass matrices—are optimized alongside the model parameters using the "reparameterization trick," making the entire process end-to-end differentiable.
Key Insights
- Entropy Transfer: HVI initializes "momentum" variables, and Hamiltonian dynamics effectively transfer entropy from these auxiliary variables into the latent variables $z$. This allows the model to capture complex, multi-modal shapes far beyond standard Gaussian approximations.
- Optimization over Asymptotics: Unlike traditional MCMC, HVI allows for the explicit optimization of the chain's parameters to reach the posterior as quickly as possible.
- Differentiable Chains: By omitting the Metropolis-Hastings rejection step, the authors maintain a fully differentiable algorithm, which they found still effectively optimizes the lower bound.
Experimental Results
The authors evaluated HVI on several benchmarks, most notably the MNIST handwritten digit dataset using both fully-connected and convolutional Variational Autoencoders (VAEs).
As shown in the results, adding HMC steps consistently improves the marginal log-likelihood. For a fully-connected VAE with an inference network, increasing from 0 to 8 leapfrog steps improved the bound from -94.18 to -88.30 nats. The "HVI + convolutional VAE" reached -81.94 nats, rivaling the performance of significantly more complex recurrent architectures like DRAW (-80.97) available at the time.
Conclusion and Future Work
HVI offers a powerful way to trade computation for accuracy in Bayesian inference. While it introduces computational overhead—each training iteration is roughly $m \times k$ times more expensive due to gradient calculations—it provides a principled path toward more expressive posterior approximations. Future work involves scaling these methods to higher-dimensional latent spaces and refining the architecture of the inverse models required for the auxiliary bound.
Summary updated