The Thermodynamic Variational Objective

Vaden Masrani, Tuan Anh Le, Frank Wood

Introduction

The Thermodynamic Variational Objective (TVO) is a novel framework for training deep generative models that bridges the gap between variational inference and thermodynamic integration. It provides a tighter lower bound on the log marginal likelihood than the standard Evidence Lower Bound (ELBO) and offers a low-variance gradient estimator that works for both continuous and discrete latent variables—without needing the traditional "reparameterization trick."

Training deep latent variable models, such as Variational Autoencoders (VAEs), involves maximizing the "evidence" (the probability of the data under the model). Because direct calculation is usually intractable, researchers use the ELBO as a proxy. However, the ELBO has two major weaknesses:

  1. Looseness: It is often a loose approximation, leading to sub-optimal model learning.
  2. Gradient Estimation: For models with discrete variables (like categories or binary choices), calculating gradients is notoriously difficult. The standard "REINFORCE" method suffers from extremely high variance, making training unstable.

While methods like the Importance Weighted Autoencoder (IWAE) tighten the bound, they can inadvertently degrade the performance of the "inference network" (the component that guesses the latent variables).

The Thermodynamic Variational Objective (TVO)

The authors introduce the TVO by framing the log evidence as a one-dimensional integral along a "geometric path" between the inference network and the true model.

  • The Path: They define a series of intermediate distributions $\pi_\beta$ that transition from the simple inference network ($\beta=0$) to the complex model posterior ($\beta=1$).
  • The Integral: Leveraging Thermodynamic Integration (TI) from physics, they show that the log evidence is the integral of the "instantaneous ELBO" across this path.
  • The Approximation: The TVO is a Riemann sum approximation of this integral. While the standard ELBO uses just one partition, the TVO uses multiple partitions ($K$), making the bound much tighter.
ELBO vs TVO integration

To optimize this objective, the authors derive a novel covariance gradient estimator. Unlike the reparameterization trick used in standard VAEs, this estimator relies on the covariance between the log-likelihood and the gradient of the log-probability. This critical difference makes it applicable to any distribution, including discrete ones.

Key Insights

  • ELBO is a Special Case: The standard ELBO is mathematically equivalent to a "left Riemann sum" with a single partition ($K=1$). Adding more partitions naturally tightens the bound.
  • The "Sweet Spot" for Partitions: Increasing the number of partitions ($K$) improves the bound but introduces noise from importance sampling. There is an optimal, often small, number of partitions (e.g., $K=2$ or $5$) that provides the best balance for learning.
  • Unifying Objectives: The TVO provides a single mathematical umbrella for several famous algorithms. For instance, the "Wake-Sleep" algorithm can be viewed as alternating between the left and right endpoints of the TVO integral.

Experimental Results

The researchers evaluated the TVO using the binarized MNIST dataset across two architectures: Sigmoid Belief Networks (SBNs) for discrete variables, and continuous latent variable models similar to standard VAEs.

Training convergence of TVO vs baselines
  • Discrete Models: In SBNs, the TVO outperformed state-of-the-art methods like VIMCO and Reweighted Wake-Sleep (RWS) in both convergence speed and final test log-evidence.
  • Continuous Models: Even without the reparameterization trick (which typically gives continuous models a massive advantage), the TVO was competitive with IWAE and significantly outperformed the standard VAE.
  • Inference Quality: Unlike IWAE, which often degrades the inference network as the bound gets tighter, the TVO successfully improved the quality of the inference network, as measured by KL divergence.

Limitations and Future Work

The primary limitation of the TVO is the reliance on manually chosen discretization points ($\beta_{1:K}$). The objective is highly sensitive to the placement of these points, especially near $\beta=0$ where the curve is steepest.

Future work aims to address this by developing adaptive grids that allow the model to automatically learn the optimal positions for the $\beta$ partitions during training. Additionally, researchers plan to explore Bayesian Quadrature—using more advanced numerical integration techniques instead of simple Riemann sums—to further tighten the bound with fewer partitions, and to apply the TVO to non-amortized variational inference and more complex structured models.

Summary updated

Abstract

We introduce the thermodynamic variational objective (TVO) for learning in both continuous and discrete deep generative models. The TVO arises from a key connection between variational inference and thermodynamic integration that results in a tighter lower bound to the log marginal likelihood than the standard variational variational evidence lower bound (ELBO) while remaining as broadly applicable. We provide a computationally efficient gradient estimator for the TVO that applies to continuous, discrete, and non-reparameterizable distributions and show that the objective functions used in variational inference, variational autoencoders, wake sleep, and inference compilation are all special cases of the TVO. We use the TVO to learn both discrete and continuous deep generative models and empirically demonstrate state of the art model and inference network learning.
BibTeX
@inproceedings{masrani2019thermodynamic,
  author = {Vaden Masrani and Tuan Anh Le and Frank Wood},
  title = {{The Thermodynamic Variational Objective}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2019},
  url = {https://openreview.net/forum?id=h3Px8ghorky},
}
Figure