Hamiltonian Variational Auto-Encoder

Anthony L. Caterini, Arnaud Doucet, Dino Sejdinovic

Introduction

Variational Auto-Encoders (VAEs) are foundational to modern generative modeling, but they typically rely on simple, "mean-field" Gaussian distributions to approximate the posterior of latent variables. This simplicity severely limits their ability to capture complex, multi-modal distributions.

To address this, researchers often employ Normalizing Flows (NFs)—sequences of invertible transformations that warp a simple distribution into a complex one. However, standard NFs are "blind" to the target distribution; they learn a transformation without explicitly utilizing information about the data's likelihood during the flow. Alternatively, using Markov Chain Monte Carlo (MCMC) techniques like Hamiltonian Monte Carlo (HMC) inside VAEs provides "target-aware" samples. Unfortunately, calculating the Evidence Lower Bound (ELBO) in these setups usually requires defining a "reverse" path for the MCMC chain. Learning these reverse kernels is notoriously difficult, adds noise, and often breaks the "reparameterization trick" essential for efficient VAE training.

The Hamiltonian Variational Auto-Encoder (HVAE)

To bridge this gap, researchers introduce the Hamiltonian Variational Auto-Encoder (HVAE), which leverages Hamiltonian Importance Sampling (HIS) to incorporate HMC dynamics directly into the inference process.

The HVAE introduces several key technical innovations:

  • Time-Inhomogeneous Dynamics (Tempering): Instead of standard HMC, HVAE uses dynamics that change over time. The sampler starts "hot" for broad exploration and "cools down" to refine near high-probability modes.
  • Optimal Reverse Kernels: By utilizing deterministic Hamiltonian dynamics (leapfrog integration), the "reverse" path is mathematically determined. This eliminates the need to learn auxiliary reverse models, solving a major pain point in prior Hamiltonian Variational Inference (HVI) work.
  • Target-Informed Flow: The latent variable transformation explicitly uses the gradient of the model's log-likelihood. Consequently, HVAE acts as a target-informed normalizing flow that inherently "knows" where the high-probability regions of the posterior reside.
  • Reparameterization-Friendly: Because the entire chain is differentiable, the model can use the reparameterization trick to obtain low-variance, unbiased gradients for both generative ($\theta$) and variational ($\phi$) parameters.

Experimental Results

The HVAE was evaluated on two primary benchmarks to test parameter estimation and generative performance.

1. Synthetic Gaussian Model A toy model was designed to test the accuracy of parameter estimation in high-dimensional spaces (up to 300 dimensions). As dimensionality increases, HVAE significantly outperforms standard Variational Bayes (VB) and standard Normalizing Flows (NF) in estimating the true model parameters.

Error in parameter estimation vs. dimensionality for HVAE compared to Variational Bayes (VB) and Normalizing Flows (NF).

2. Binarized MNIST On the standard $28 \times 28$ handwritten digits dataset, HVAE achieved a Negative Log-Likelihood (NLL) of approximately 82.62. This outperforms the base VAE (83.20) and provides results highly competitive with much more complex, parameter-heavy normalizing flow architectures. Ablation studies further confirmed that the time-inhomogeneous (tempered) dynamics are strictly superior to homogeneous dynamics for this architecture.

Key Insights and Limitations

The success of HVAE stems from the realization that the "reverse" is free. In deterministic systems like leapfrog integration, the optimal reverse kernel is simply the inverse mapping. Because it is reversible and has a tractable Jacobian, ELBO calculations become vastly simpler and more stable. Furthermore, because the flow is guided by the target distribution's gradients, HVAE achieves better results with fewer parameters than "black-box" NFs.

However, this target-awareness comes with computational overhead. HVAE requires calculating the gradient of the log-likelihood at every leapfrog step. For large neural networks, this significantly increases training time and memory usage per iteration. Backpropagating through many Hamiltonian steps limits the practical "depth" of the flow. Future work may explore alternative physics-inspired dynamics, such as the Nosé-Hoover thermostat, to further simplify Jacobian calculations and improve sampling efficiency.

Summary updated

Abstract

Variational Auto-Encoders (VAE) have become very popular techniques to perform inference and learning in latent variable models as they allow us to leverage the rich representational power of neural networks to obtain flexible approximations of the posterior of latent variables as well as tight evidence lower bounds (ELBO). Com- bined with stochastic variational inference, this provides a methodology scaling to large datasets. However, for this methodology to be practically efficient, it is neces- sary to obtain low-variance unbiased estimators of the ELBO and its gradients with respect to the parameters of interest. While the use of Markov chain Monte Carlo (MCMC) techniques such as Hamiltonian Monte Carlo (HMC) has been previously suggested to achieve this [23, 26], the proposed methods require specifying reverse kernels which have a large impact on performance. Additionally, the resulting unbiased estimator of the ELBO for most MCMC kernels is typically not amenable to the reparameterization trick. We show here how to optimally select reverse kernels in this setting and, by building upon Hamiltonian Importance Sampling (HIS) [17], we obtain a scheme that provides low-variance unbiased estimators of the ELBO and its gradients using the reparameterization trick. This allows us to develop a Hamiltonian Variational Auto-Encoder (HVAE). This method can be re-interpreted as a target-informed normalizing flow [20] which, within our context, only requires a few evaluations of the gradient of the sampled likelihood and trivial Jacobian calculations at each iteration.
BibTeX
@inproceedings{caterini2018hamiltonian,
  author = {Anthony L. Caterini and Arnaud Doucet and Dino Sejdinovic},
  title = {{Hamiltonian Variational Auto-Encoder}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2018},
  url = {https://openreview.net/forum?id=BJNRyY-d-r},
}
Figure