Using Large Ensembles of Control Variates for Variational Inference

Tomas Geffner, Justin Domke

Abstract

Variational inference is increasingly being addressed with stochastic optimization. In this setting, the gradient's variance plays a crucial role in the optimization procedure, since high variance gradients lead to poor convergence. A popular approach used to reduce gradient's variance involves the use of control variates. Despite the good results obtained, control variates developed for variational inference are typically looked at in isolation. In this paper we clarify the large number of control variates that are available by giving a systematic view of how they are derived. We also present a Bayesian risk minimization framework in which the quality of a procedure for combining control variates is quantified by its effect on optimization convergence rates, which leads to a very simple combination rule. Results show that combining a large number of control variates this way significantly improves the convergence of inference over using the typical gradient estimators or a reduced number of control variates.

Introduction

Modern machine learning often relies on Variational Inference (VI) to approximate complex, intractable probability distributions. A common strategy in VI is to optimize the Evidence Lower Bound (ELBO) using stochastic gradient descent. However, the efficiency of this process is heavily dependent on the variance of the gradient estimates. High variance leads to unstable optimization, requiring smaller learning rates and significantly more iterations to reach convergence.

To mitigate this, researchers often employ control variates (CVs)—statistical terms with zero expectation that are correlated with the gradient noise. While various CVs have been proposed, they are frequently evaluated in isolation. This work provides a unified framework for deriving a wide array of CVs and introduces a robust Bayesian method for ensembling them to maximize variance reduction.

Methodology

A Systematic Framework for Control Variate Derivation

The authors demonstrate that the gradient of the ELBO can be decomposed into four primary components: Data, Prior, Variational, and Score. By applying different estimation techniques (such as the Score Function estimator versus the Reparameterization trick) to these components and taking their difference, one can systematically generate valid control variates. Because both estimators target the same underlying value, their difference has an expected value of zero, fulfilling the fundamental requirement for a control variate.

Regularized Ensembling via Bayesian Decision Theory

While combining multiple CVs can theoretically lead to greater variance reduction, estimating the optimal weights for these combinations is challenging. Naive estimation from small minibatches often introduces new noise, potentially worsening performance.

To address this, the authors propose a Bayesian risk minimization framework. By placing a Normal-Wishart prior over the gradients and CVs, they derive a regularized combination rule. This approach treats weight estimation as a decision problem under uncertainty, effectively "shrinking" the weights when data is scarce to prevent overfitting to noisy minibatch statistics. Furthermore, the use of exponential moving averages allows the algorithm to accumulate information across iterations, providing more stable weight estimates without significant computational overhead.

Experimental Results

The proposed method was evaluated using Bayesian binary logistic regression on three UCI benchmark datasets: Ionosphere, Australian, and Sonar.

Variance Reduction and Convergence

The primary benefit of the ensemble approach is a dramatic reduction in gradient variance. As shown in Figure 1, the expected squared norm of the gradient—a proxy for variance—decreases much more rapidly when using the full ensemble of control variates compared to the base gradient or even the best single control variate.

Comparison of gradient variance (expected squared norm) over iterations for the base gradient, the best single control variate, and the combined ensemble.

Optimization Performance

The reduction in variance translates directly into faster and more stable convergence to higher ELBO values. Across all tested datasets and various learning rates, the "All CVs combined" approach consistently outperformed standard methods.

ELBO convergence across three datasets (Ionosphere, Australian, Sonar) and various learning rates, comparing different control variate combinations against the base gradient.

Notably, the ensemble method exhibited greater robustness to the choice of learning rate. In scenarios where high learning rates caused standard gradient estimators to diverge or stall, the regularized ensemble remained stable, allowing for faster training. For instance, on the Sonar dataset, the full ensemble achieved a final ELBO of approximately -117, whereas the base gradient estimator struggled to reach -442 under similar conditions.

Conclusion

This paper unifies the landscape of control variates in Variational Inference and provides a principled Bayesian framework for their combination. By ensembling a large number of CVs with proper regularization, practitioners can achieve significantly lower gradient variance, leading to faster convergence and increased robustness to hyperparameter settings. Future work may explore automated selection of CV subsets to further optimize the trade-off between variance reduction and computational cost.

Summary updated

Abstract

Variational inference is increasingly being addressed with stochastic optimization. In this setting, the gradient's variance plays a crucial role in the optimization procedure, since high variance gradients lead to poor convergence. A popular approach used to reduce gradient's variance involves the use of control variates. Despite the good results obtained, control variates developed for variational inference are typically looked at in isolation. In this paper we clarify the large number of control variates that are available by giving a systematic view of how they are derived. We also present a Bayesian risk minimization framework in which the quality of a procedure for combining control variates is quantified by its effect on optimization convergence rates, which leads to a very simple combination rule. Results show that combining a large number of control variates this way significantly improves the convergence of inference over using the typical gradient estimators or a reduced number of control variates.
BibTeX
@inproceedings{geffner2018using,
  author = {Tomas Geffner and Justin Domke},
  title = {{Using Large Ensembles of Control Variates for Variational Inference}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2018},
  url = {https://openreview.net/forum?id=SyWYuPZuZH},
}
Figure