This paper, presented at ICML 2017 by Justin Domke, provides a theoretical bridge between the two pillars of approximate Bayesian inference: Markov Chain Monte Carlo (MCMC) and Variational Inference (VI). By deriving a new divergence bound, the author creates a "smooth continuum" of algorithms that allow practitioners to trade off the speed of VI with the asymptotic accuracy of MCMC.
The Performance Gap in Bayesian Inference
In Bayesian inference, we often need to sample from a complex posterior distribution $p(z)$. Historically, practitioners have had to choose between two distinct paradigms:
- MCMC (e.g., Langevin Dynamics): Highly accurate in the long run, but often painfully slow to converge.
- Variational Inference (VI): Very fast to find a "good enough" approximation, but fundamentally limited by the simplicity of the chosen distribution family (known as the "variational gap").
There is a massive performance gap between these two approaches. If you have more computational time than VI needs, but less than MCMC requires to converge, there hasn't been a principled way to utilize that extra time to get a better answer. This paper addresses the lack of a fine-grained tradeoff between these two regimes.
A Principled Hybrid Approach
The core contribution of the paper is a theoretical framework based on Information Theory that interpolates between Langevin Dynamics and Stochastic Gradient Variational Inference (SGVI).
- The Hybrid Bound: The paper defines a new objective function, $D_\beta$, which is a convex combination of two different upper bounds on the KL-divergence between the approximating marginal and the target distribution.
- The Interpolation Parameter ($\beta$): A single parameter controls the behavior of the algorithm. When $eta = 0$, the objective recovers standard Variational Inference. When $eta = 1$, the objective and resulting dynamics recover Langevin Dynamics (MCMC).
- Langevin Dynamics on Parameters: Instead of running MCMC directly on the variables of interest ($z$), the algorithm runs a stochastic gradient-based MCMC on the parameters ($w$) of a variational distribution $q(z|w)$.
As seen in the toy distributions above, the algorithm smoothly transitions from a single-mode variational fit ($\beta=0$) to a full, noisy particle-like coverage of the posterior ($\beta=1$). The author notes that MCMC creates "entropy" (coverage of the distribution) through the randomness of the Markov chain, while VI creates it through the explicit width of the variational distribution. The hybrid allows for "fractions" of both strategies.
Key Results: The Pareto Frontier
The paper evaluates the hybrid algorithm on real-world binary classification datasets (including a1a, australian, ionosphere, and sonar from the UCI repository and LibSVM) using Bayesian Logistic Regression.
In many practical scenarios, the "best" algorithm depends entirely on your time budget. The experiments demonstrate that intermediate values of $\beta$ (e.g., 0.4 or 0.6) often sit on the "Pareto frontier"—meaning they provide the lowest error for a given amount of computation time.
On these tasks, the hybrid algorithm frequently outperformed both pure SGVI and pure Langevin Dynamics. Specifically, for a wide range of iteration counts (e.g., $10^3$ to $10^5$ iterations), intermediate $\beta$ values achieved lower Maximum Mean Discrepancy (MMD), a metric measuring how close the generated samples are to the true posterior.
Limitations and Future Work
While the framework is robust and mathematically sound, the author identifies a few areas for future exploration:
- Base Measure Choice: The performance was surprisingly insensitive to the specific choice of the "base measure" $r(w)$, but optimizing this at runtime could lead to even tighter bounds.
- Simple Variational Families: The experiments primarily used fully-factorized Gaussian distributions. Applying this framework to more complex families, like Normalizing Flows or structured Gaussians, remains an open area for research.
- Step-size Sensitivity: Like all Langevin-based methods, the algorithm is sensitive to the step-size $\epsilon$. While the paper retrospectively chose the best $\epsilon$ for its evaluations, a practical implementation would require a more automated tuning schedule.
Ultimately, this paper provides a rigorous, principled derivation for hybridizing MCMC and VI, offering a valuable new tool for practitioners navigating the tradeoff between inference speed and accuracy.
Summary updated