The Thermodynamic Variational Objective

Vaden Masrani, Tuan Anh Le, Frank Wood

TL;DR

The paper introduces the Thermodynamic Variational Objective (TVO), a novel framework that bridges variational inference and thermodynamic integration to provide a tighter lower bound on the log marginal likelihood. Crucially, TVO works seamlessly for both continuous and discrete latent variables without requiring the reparameterization trick, outperforming or competing closely with state-of-the-art generative modeling objectives.

What Problem Does This Paper Address?

Deep latent variable models (like Variational Autoencoders, or VAEs) are powerful tools for unsupervised learning, but training them is notoriously difficult. The standard approach maximizes the Evidence Lower Bound (ELBO), which can often be quite loose, leading to sub-optimal model learning.

While alternative objectives like the Importance Weighted Autoencoder (IWAE) succeed in tightening this bound, they suffer from a nasty side effect: tightening the bound can actually degrade the quality of the inference network. Furthermore, when dealing with discrete latent variables, researchers must rely on high-variance gradient estimators (like REINFORCE) or complex continuous relaxations. The authors set out to find a tighter, universally applicable objective that avoids these pitfalls.

Methods

The core innovation of this paper is connecting Variational Inference (VI) to Thermodynamic Integration (TI)—a technique borrowed from physics used to compute the ratio of normalizing constants.

By constructing a geometric path parameterized by a scalar $\beta \in [0,1]$ between the variational distribution $q_\phi(\mathbf{z}|\mathbf{x})$ (at $\beta=0$) and the true joint distribution $p_\theta(\mathbf{x},\mathbf{z})$ (at $\beta=1$), the authors show that the exact log evidence can be written as a continuous 1D integral of the "instantaneous ELBO" over the unit interval.

Figure 1: The thermodynamic variational objective (TVO) (center) is a finite sum numerical approxi- mation to log pθ(x), defined by the thermodynamic variational identity (TVI) (right). The ELBO (left) is a single partition approximation of the same integral. πβ is given in (7)

To turn this continuous identity into a practical objective, the TVO approximates the integral using a left Riemann sum with $K$ partitions. To optimize this without needing the reparameterization trick, the authors derive a novel score-function-based covariance gradient estimator. By leveraging importance sampling, they can reuse a single set of samples across all $\beta$ partitions, making the objective computationally efficient and significantly reducing gradient variance.

Datasets

The authors evaluate the TVO on the standard binarized MNIST dataset, utilizing the classic 50k/10k/10k train/validation/test split to evaluate both discrete and continuous deep generative models.

Key Insights and Ideas

  • ELBO as a Special Case: Conceptually, the standard ELBO is revealed to be nothing more than a single-partition, left Riemann sum approximation of this broader thermodynamic identity. Similarly, other popular objectives like Wake-Sleep and Inference Compilation emerge as special cases at the endpoints of the thermodynamic path.
  • The Partition "Sweet Spot": While mathematically a higher number of partitions ($K$) yields a tighter mathematical bound, it also introduces more importance sampling noise. The authors discover a distinct "sweet spot" for $K$, beyond which performance begins to degrade.
  • Exploiting Curvature ($\beta^*$): The integrand curve typically rises sharply at a specific point of maximum curvature ($\beta^*$) before flattening out. If the first partition ($\beta_1$) is strategically placed right around this elbow, the TVO can achieve excellent model learning with as few as $K=2$ partitions.
Figure 4: The location of β∗, the point of maximum curvature.

Key Results

  • Discrete Latent Variables: For discrete models, TVO consistently outperforms state-of-the-art methods like VIMCO and Reweighted Wake-Sleep (RWS) when the number of samples $S < 50$. It converges faster and achieves a higher final test log-evidence. Furthermore, it achieves the lowest KL divergence between the inference network and the true posterior.
Figure 3: Comparisons with baselines on a held out test set. (Left) Learning curves for different methods. For TVO outperforms other methods both in terms of speed of convergence and the learned model for S < 50. At S = 50 VIMCO achieves a higher test log evidence but takes longer to converge than the TVO. (Right) KL divergence between current q and p (which measures how well q “tracks” p) is lowest for TVO.
  • Continuous Latent Variables: Despite not using the reparameterization trick, TVO outperforms standard VAEs and performs competitively with IWAE at 50 samples.
  • Gradient Efficiency: The proposed covariance gradient estimator exhibits lower variance than VIMCO and RWS as training progresses, proving that variance reduction can be achieved naturally through thermodynamic principles rather than ad-hoc control variates.

Limitations and Future Work

The primary limitation of the TVO framework is its sensitivity to the discretization schedule ($\beta_{1:K}$). Currently, finding the optimal spacing requires a manual hyperparameter search; if $\beta_1$ is placed poorly, the Riemann sum fails to capture the curve's sharp initial rise.

The authors suggest that future work should explore adaptive discretization schedules, potentially using techniques from Bayesian numerical quadrature to automatically find the optimal partition locations during training.

Summary updated

Abstract

We introduce the thermodynamic variational objective (TVO) for learning in both continuous and discrete deep generative models. The TVO arises from a key connection between variational inference and thermodynamic integration that results in a tighter lower bound to the log marginal likelihood than the standard variational variational evidence lower bound (ELBO) while remaining as broadly applicable. We provide a computationally efficient gradient estimator for the TVO that applies to continuous, discrete, and non-reparameterizable distributions and show that the objective functions used in variational inference, variational autoencoders, wake sleep, and inference compilation are all special cases of the TVO. We use the TVO to learn both discrete and continuous deep generative models and empirically demonstrate state of the art model and inference network learning.
BibTeX
@inproceedings{masrani2019thermodynamic,
  author = {Vaden Masrani and Tuan Anh Le and Frank Wood},
  title = {{The Thermodynamic Variational Objective}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2019},
  url = {https://openreview.net/forum?id=h3Px8ghorky},
}

Loading related content...

Figure