TL;DR
This paper introduces a novel control variate technique that significantly reduces the variance of reparameterization gradients in Variational Inference (VI). By using a learned quadratic approximation of the model and a "double-descent" optimization scheme, the authors achieve orders-of-magnitude variance reduction for complex, non-factorized distributions where previous methods have historically failed.
What Problem Does This Paper Address?
In Bayesian machine learning, Stochastic Gradient Variational Inference (SGVI) is frequently used to approximate complex posterior distributions. The "reparameterization trick" is the gold standard for estimating the gradients needed for this optimization. However, these estimates are often noisy and suffer from high variance, which leads to slow convergence or outright optimization failure.
While variance reduction techniques like "control variates" exist, they have a major blind spot. Previous state-of-the-art methods—such as Taylor-expansion-based control variates—work well for the mean parameters of a distribution but are notoriously ineffective for scale (covariance) parameters. This is a deal-breaker for modern VI, where practitioners want to move beyond simple "mean-field" (diagonal) approximations toward more powerful full-rank or structured covariance distributions. In these complex cases, the scale parameters actually contribute the most noise, rendering traditional Taylor-based methods nearly useless.
Methods
The authors propose a new control variate based on a quadratic approximation ($\hat{f}$) of the model's log-likelihood ($f$).
- The Control Variate: A control variate is a random variable with a known expectation of zero that is added to an estimator to cancel out noise. The authors define theirs as the difference between the exact gradient of the quadratic approximation and a stochastic estimate of that same gradient.
- Tractability: They prove that this control variate is computationally tractable for any distribution where the mean and covariance are known (e.g., Gaussians, Student-t, or Householder flows).
- Double-Descent Optimization: Instead of using a fixed Taylor expansion, they learn the parameters of the quadratic approximation. They use a "double-descent" scheme: in each iteration, they update the variational parameters to maximize the Evidence Lower Bound (ELBO) and simultaneously update the quadratic approximation to minimize the gradient's variance.
- Efficiency: They propose a "proxy objective" for fitting the quadratic that avoids expensive Hessian calculations, making the method much faster than second-order Taylor approaches.
Key Insights and Ideas
- The "Scale" Failure: The paper provides a sharp analysis of why previous Taylor-based methods fail. For scale parameters, Taylor methods essentially reduce to a "constant" (zero-order) approximation, which provides zero variance reduction.
- Learning vs. Expanding: By learning the quadratic approximation globally over the distribution's current "footprint" rather than just at a single point (like a Taylor expansion), the control variate remains effective even when the distribution has high variance.
- Variance Dominance: In non-factorized distributions (like full-rank Gaussians), the scale parameters dominate the total gradient variance. This explains why the authors' method shows such dramatic improvements over baselines that only handle mean parameters well.
Key Results
The researchers evaluated their method on three standard probabilistic modeling tasks: Bayesian Logistic Regression, Hierarchical Regression, and Bayesian Neural Networks (BNN).
- Massive Variance Reduction: In experiments with full-rank and low-rank Gaussians, the proposed control variate reduced gradient variance by several orders of magnitude compared to both the "Base" reparameterization and Taylor-based methods.
- Faster Convergence: The method reaches higher ELBO values in significantly fewer iterations. For example, in the BNN task, the proposed method with only 10 samples outperformed the base estimator using 50 samples.
- Robustness: The method is much less sensitive to the choice of learning rate. While standard SGVI often diverges with larger step sizes, this approach remains stable, allowing for even faster optimization.
- Computational Overhead: While adding a control variate increases the cost per iteration (roughly 50% higher than the base estimator), the authors show that their method is significantly faster than a second-order Taylor approach. More importantly, this slight per-iteration overhead is easily offset by the massive gains in overall convergence speed.
Limitations and Future Work
- Known Moments: The method requires the variational distribution to have a known mean and covariance. While this covers most common distributions, it might not apply to some highly exotic or implicit distributions.
- Quadratic Limits: A quadratic is a "global" approximation. If the underlying model ($f$) is extremely non-linear or multi-modal within the region of the variational distribution, a quadratic might not be flexible enough to capture the gradient behavior perfectly.
- Future Directions: The authors suggest that this framework could be extended to other types of gradient estimators, such as the score function estimator, or combined with other advanced techniques like "sticking-the-landing" estimators.
Summary updated