Abstract
The Thermodynamic Variational Objective (TVO) bridges the gap between Variational Inference (VI) and Thermodynamic Integration (TI) to provide a tighter, more flexible lower bound for training deep generative models. By expressing the log marginal likelihood as a one-dimensional integral along a geometric path of distributions, the TVO enables a Riemann sum approximation that unifies several existing variational objectives. Furthermore, the introduction of a novel covariance gradient estimator allows for stable training of models with both discrete and continuous latent variables without requiring the reparameterization trick.
Introduction
Training deep generative models typically involves maximizing the log marginal likelihood of the observed data, $\log p_\theta(\mathbf{x})$. Because this quantity is often intractable, researchers rely on the Evidence Lower Bound (ELBO) as a surrogate objective. While the ELBO is widely used, it suffers from two primary limitations: the "tighter bounds" effect, where increasing the number of samples can paradoxically degrade the learning of the inference network, and the difficulty of optimizing models with discrete latent variables, which often necessitates high-variance gradient estimators like REINFORCE.
The Thermodynamic Variational Identity
The authors propose a solution rooted in statistical physics. They define a path of distributions $\pi_\beta(\mathbf{z}|\mathbf{x})$ that interpolates between the variational posterior $q_\phi(\mathbf{z}|\mathbf{x})$ at $\beta=0$ and the true posterior $p_\theta(\mathbf{z}|\mathbf{x})$ at $\beta=1$. This leads to the Thermodynamic Variational Identity (TVI), which states that the log evidence is the integral of the expected log-likelihood ratio over this path: $$\log p_\theta(\mathbf{x}) = \int_0^1 \mathbb{E}_{\pi_\beta} \left[ \log \frac{p_\theta(\mathbf{x}, \mathbf{z})}{q_\phi(\mathbf{z}|\mathbf{x})} \right] d\beta$$
The TVO Objective
Since the exact integral is intractable, the TVO approximates it using a left Riemann sum with $K$ partitions. Because the integrand is monotonically increasing with respect to $\beta$, this Riemann sum is guaranteed to be a lower bound on the log evidence.
As shown in the figure above, the standard ELBO corresponds to a single-partition TVO evaluated at $\beta=0$. By increasing the number of partitions, the TVO provides a progressively tighter bound. Crucially, the TVO framework unifies disparate algorithms: the Evidence Upper Bound (EUBO) is the right endpoint at $\beta=1$, while Wake-Sleep and its variants can be viewed as special cases at the path's endpoints.
Optimization and Gradient Estimation
To optimize the TVO, the authors derive a covariance gradient estimator. Unlike the reparameterization trick, which is limited to continuous variables, or REINFORCE, which can be unstable, the covariance estimator uses importance sampling to compute gradients for both $\theta$ and $\phi$. By reusing the same batch of samples across all $\beta$ partitions (Common Random Numbers), the TVO achieves high efficiency; increasing the number of partitions $K$ adds negligible computational overhead.
Empirical Results
The TVO was evaluated on binarized MNIST using both discrete Sigmoid Belief Networks and continuous models.
As illustrated in the training curves, the TVO (red) consistently achieves higher log-evidence and lower KL divergence between the variational and true posterior compared to Reweighted Wake-Sleep (RWS) and VIMCO. This confirms that the TVO not only tightens the bound but also improves the quality of the inference network, avoiding the "tighter bounds" effect seen in IWAE. In continuous models, the TVO remains highly competitive with IWAE despite not using the reparameterization trick.
Conclusion and Future Work
The Thermodynamic Variational Objective offers a powerful and unifying framework for variational inference. By leveraging thermodynamic integration, it provides a tighter bound and more stable gradients for a wide class of generative models. While the current implementation is sensitive to the choice of discretization points ($\beta_k$), future work involving adaptive integration schemes or Bayesian numerical quadrature promises to make the TVO even more robust and easier to deploy.
Summary updated