Deep Markov Chain Monte Carlo

Babak Shahbaba, Luis Martinez Lomeli, Tian Chen, Shiwei LanarXiv 2019 · DOI · arXiv

Abstract

We propose a new computationally efficient sampling scheme for Bayesian inference involving high dimensional probability distributions. Our method maps the original parameter space into a low-dimensional latent space, explores the latent space to generate samples, and maps these samples back to the original space for inference. While our method can be used in conjunction with any dimension reduction technique to obtain the latent space, and any standard sampling algorithm to explore the low-dimensional space, here we specifically use a combination of auto-encoders (for dimensionality reduction) and Hamiltonian Monte Carlo (HMC, for sampling). To this end, we first run an HMC to generate some initial samples from the original parameter space, and then use these samples to train an auto-encoder. Next, starting with an initial state, we use the encoding part of the autoencoder to map the initial state to a point in the low-dimensional latent space. Using another HMC, this point is then treated as an initial state in the latent space to generate a new state, which is then mapped to the original space using the decoding part of the auto-encoder. The resulting point can be treated as a Metropolis-Hasting (MH) proposal, which is either accepted or rejected. While the induced dynamics in the parameter space is no longer Hamiltonian, it remains time reversible, and the Markov chain could still converge to the canonical distribution using a volume correction term. Dropping the volume correction step results in convergence to an approximate but reasonably accurate distribution. The empirical results based on several high-dimensional problems show that our method could substantially reduce the computational cost of Bayesian inference.

Introduction

Bayesian inference is a gold standard for robust data analysis, providing a principled framework for quantifying uncertainty. However, it faces a significant "speed limit" in high-dimensional settings. Standard Markov Chain Monte Carlo (MCMC) algorithms, such as Hamiltonian Monte Carlo (HMC), struggle when the parameter space grows to thousands of dimensions. This difficulty arises from two primary factors: the "curse of dimensionality," where the typical set becomes a needle in a high-dimensional haystack, and the prohibitive computational cost of calculating gradients for HMC at every step.

The core insight of this work is the Manifold Hypothesis: even in models with thousands of parameters, the posterior distribution often resides on a much lower-dimensional manifold. By identifying and navigating this manifold, we can bypass the inefficiencies of exploring "empty" high-dimensional space. We propose Deep MCMC, a framework that bridges the gap between the accuracy of MCMC and the speed of Variational Bayes (VB) by performing sampling in a compressed latent space.

Methodology: Auto-encoding HMC (AE-HMC)

The proposed method, AE-HMC, utilizes a three-stage workflow to optimize the sampling process:

  1. Pre-sampling: A standard HMC is run for a short duration (e.g., 1,000 samples) to obtain an initial, rough approximation of the posterior distribution.
  2. Dimensionality Reduction: An auto-encoder is trained on these initial samples. The encoder ($ϕ$) learns to map high-dimensional parameters ($q_v$) to a low-dimensional latent code ($q_h$), while the decoder ($ψ$) learns the inverse mapping to reconstruct the original parameters.
  3. Latent Sampling: Sampling is conducted in the low-dimensional latent space. The current state is encoded, a new state is proposed using HMC dynamics within the latent space, and this proposal is then decoded back to the original space. A final Metropolis-Hastings step determines the acceptance of the move.
Conceptual mapping between the high-dimensional parameter space (A) and the low-dimensional latent space (B) using encoder ϕ and decoder ψ.

To ensure mathematical exactness, a volume correction term (Jacobian) can be included. However, the authors demonstrate that an "Approximate Bayes" version, which omits this term, yields significant speed gains while maintaining sufficient accuracy for most practical applications.

Experimental Evaluation

The researchers evaluated AE-HMC across several challenging high-dimensional benchmarks:

  • Synthetic Data: Binary classification with 500 features, including high correlation.
  • Real-world Classification: Datasets including CNAE-9 (document classification), UCI Digits, and MNIST (binary classification of the first two digit groups).
  • Physics-based Inverse Problem: A high-dimensional Bayesian inverse problem involving an Elliptic Partial Differential Equation (PDE) on meshes up to 6,500+ dimensions.

Results and Discussion

AE-HMC provides a substantial reduction in computational overhead without sacrificing predictive performance. On the MNIST dataset, AE-HMC was 6.8x faster than standard HMC, while on the complex PDE inverse problem, it achieved a 15.1x speedup. Accuracy remained robust; for instance, on MNIST, AE-HMC achieved 99.2% accuracy compared to 99.6% for standard HMC.

Comparison of posterior distributions for model parameters β_i. The blue and orange curves represent standard HMC and the proposed AE-HMC, respectively.

As shown in the posterior visualizations, while the approximate AE-HMC distributions (orange) can be slightly more concentrated than the true posterior (blue), they consistently cover the true parameter values and capture the essential structure of the distribution. This trade-off—slight loss in exactness for massive gains in speed—is highly beneficial for large-scale Bayesian modeling.

Conclusion and Future Work

Deep MCMC demonstrates that latent space sampling via auto-encoders is a powerful tool for scaling Bayesian inference. By calculating gradients in a small latent space via the chain rule through the decoder, the method achieves "Variational-like" speeds while retaining the global exploration benefits of MCMC.

Future research directions include developing volume-preserving embeddings (such as flow-based models) to eliminate the need for approximate distributions and implementing automated methods for determining the optimal intrinsic dimension of the latent space. These advancements will further refine the balance between computational efficiency and posterior accuracy.

Summary updated

Abstract

We propose a new computationally efficient sampling scheme for Bayesian inference involving high dimensional probability distributions. Our method maps the original parameter space into a low-dimensional latent space, explores the latent space to generate samples, and maps these samples back to the original space for inference. While our method can be used in conjunction with any dimension reduction technique to obtain the latent space, and any standard sampling algorithm to explore the low-dimensional space, here we specifically use a combination of auto-encoders (for dimensionality reduction) and Hamiltonian Monte Carlo (HMC, for sampling). To this end, we first run an HMC to generate some initial samples from the original parameter space, and then use these samples to train an auto-encoder. Next, starting with an initial state, we use the encoding part of the autoencoder to map the initial state to a point in the low-dimensional latent space. Using another HMC, this point is then treated as an initial state in the latent space to generate a new state, which is then mapped to the original space using the decoding part of the auto-encoder. The resulting point can be treated as a Metropolis-Hasting (MH) proposal, which is either accepted or rejected. While the induced dynamics in the parameter space is no longer Hamiltonian, it remains time reversible, and the Markov chain could still converge to the canonical distribution using a volume correction term. Dropping the volume correction step results in convergence to an approximate but reasonably accurate distribution. The empirical results based on several high-dimensional problems show that our method could substantially reduce the computational cost of Bayesian inference.
BibTeX
@misc{shahbaba2019deep,
  author = {Babak Shahbaba and Luis Martinez Lomeli and Tian Chen and Shiwei Lan},
  title = {{Deep Markov Chain Monte Carlo}},
  year = {2019},
  eprint = {1910.05692},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1910.05692},
}
Figure