This paper introduces Auxiliary Variational MCMC (AVS), a framework that bridges the gap between Variational Inference (VI) and Markov Chain Monte Carlo (MCMC). By using neural networks to learn the underlying "shape" of a distribution, the authors create a sampler capable of taking massive, efficient leaps across complex probability landscapes—all without needing to calculate gradients of the target distribution.
The Problem: Speed vs. Accuracy in Sampling
In Bayesian statistics and machine learning, sampling from complex, high-dimensional distributions is a frequent necessity. Traditionally, practitioners rely on two main tools, each with significant trade-offs:
- Variational Inference (VI): Fast and computationally cheap, but it often oversimplifies the distribution, leading to biased or inaccurate results.
- MCMC: Theoretically exact, but notoriously slow. Standard methods like Random Walk Metropolis (RWM) struggle in high dimensions because they move blindly, frequently stepping into "empty" regions of low probability and rejecting the move.
More advanced MCMC methods, like Hamiltonian Monte Carlo (HMC), use gradients to navigate efficiently. However, gradients can be computationally expensive or entirely impossible to compute for certain models. The authors address the need for a sampler that is as exact as MCMC but as "smart" about the distribution's structure as VI, without being tethered to gradient calculations.
Methods: The Auxiliary Variational Sampler
AVS uses deep learning to map high-dimensional probability distributions onto a simpler, low-dimensional latent space. By performing a random walk in this "shortcut" space and mapping back, the sampler suppresses inefficient random-walk behavior. It works in two distinct phases:
1. The Training Phase The authors use the "auxiliary variational method" to fit a model to the target distribution $p(x)$. They introduce a low-dimensional latent variable $a$ (the "auxiliary" space). Using neural networks, they learn three key mappings:
- $q(a|x)$: Mapping from the high-dimensional data to the low-dimensional latent space.
- $q(x|a)$: Mapping from the latent space back to the data space.
- $q(a)$: The distribution of the latent variables.
2. The Sampling Phase Once trained, the MCMC sampler doesn't just move randomly in the original data space $x$. Instead, it:
- Maps the current point $x$ to the latent space $a$.
- Takes a small, random step in the latent space (an Auxiliary Random Walk).
- Maps that new latent point back to the data space to propose a new $x'$.
- Accepts or rejects the move using a modified Metropolis-Hastings rule.
Because the latent space is low-dimensional and captures the "manifold" where the probability is high, a small step in $a$ can correspond to a very large, successful move in $x$.
Key Insights and Results
The authors evaluated AVS on several challenging benchmarks, including a 2D "Ring Density" (testing the ability to follow a constrained manifold), a Mixture of Gaussians (testing mode-hopping), a Mixture of Student T distributions, and a real-world Bayesian Logistic Regression task using the Heart dataset.
- Exploiting Low-Dimensionality: Most high-dimensional distributions actually live on low-dimensional manifolds. By forcing the auxiliary variable $a$ to be low-dimensional, the neural network learns a compressed representation of the distribution's structure.
- Gradient-Free Efficiency: Unlike HMC, AVS does not require the gradient of the target distribution during the sampling phase, making it a versatile "black-box" sampler.
- Mode Hopping as a Shortcut: In the latent space, two distant modes in the data space might be placed very close together. This allows the sampler to "teleport" between modes in a single step, solving a classic MCMC bottleneck. In the Mixture of Gaussians test, RWM and HMC got stuck in a single mode, while AVS jumped back and forth effortlessly.
Ultimately, AVS significantly outperformed Random Walk Metropolis across all tests. In the Ring and Mixture of Student T experiments, it provided much higher Effective Sample Size (ESS) than standard HMC. When accounting for both training and sampling time, AVS was often orders of magnitude more efficient than other state-of-the-art neural samplers on moderately sized problems.
Limitations and Future Work
While powerful, AVS has a few limitations:
- The "Missed Mode" Problem: AVS relies heavily on the initial variational fit. If the neural network misses a mode entirely during the training phase, the sampler is highly unlikely to ever find it during the sampling phase.
- Scaling to Huge Data: The authors note that the training overhead might become a bottleneck for extremely large-scale datasets.
Future work could involve "iterative improvement," where the MCMC samples generated by AVS are used to further refine the variational model in a continuous feedback loop, potentially mitigating the missed mode problem.
Summary updated