Abstract
Approximate inference in complex probabilistic models such as deep Gaussian processes requires the optimisation of doubly stochastic objective functions. These objectives incorporate randomness from both mini-batching and Monte Carlo sampling. While this enables scalability, it introduces significant gradient noise that can hinder convergence. This paper introduces a method to significantly reduce gradient noise by using a small "recognition network" to learn context-aware control variates. By predicting optimal control variate coefficients for each specific data point, the proposed approach achieves superior variance reduction and better convergence compared to standard, batch-independent methods.
Introduction
Modern machine learning models often rely on "doubly stochastic" objectives. These functions are characterized by two primary sources of randomness: mini-batching, where only a subset of data is processed at each step, and Monte Carlo sampling, used to approximate complex integrals in models like Variational Auto-Encoders (VAEs) or Gaussian Processes (GPs). While these techniques are essential for large-scale learning, the resulting gradient variance can lead to slow convergence or instability.
Standard variance reduction techniques employ control variates—terms with zero expectation that are correlated with the gradient noise to cancel it out. However, traditional methods often use a single, global control variate for an entire batch. This paper argues that such "one-size-fits-all" approaches are sub-optimal because different data points respond differently to noise.
Amortized Control Variates
The core contribution of this work is the concept of Amortized Control Variates. Instead of a global coefficient, the authors propose using a secondary "recognition network" ($r_\phi$) that takes a data point (the "context") as input and outputs the optimal control variate coefficients for that specific instance.
As shown in the figure above, the relationship between randomness ($\epsilon$) and the gradient ($g$) is context-dependent. A "context-free" control variate must average across all possible data points, which limits its effectiveness for any specific mini-batch. By amortizing the computation of these coefficients through a neural network, the method provides a scalable way to adapt to individual samples.
Training Objectives
Training the recognition network efficiently is crucial. The authors propose three estimators:
- Partial Gradients Estimator: Highly accurate but computationally expensive.
- Gradient Sum Estimator (GS): A more efficient alternative using the sum of gradients across the batch.
- Squared Difference Estimator (SD): A fast regression-based approach that matches the control variate to the gradient.
Experimental Evaluation
The method was evaluated on two distinct tasks: Bayesian Logistic Regression using the Titanic dataset and Deep Gaussian Processes (DGP) using the Airfoil Self-Noise dataset. These benchmarks test the approach on both simple linear models and complex, non-linear architectures.
Key Results
The amortized approach consistently outperformed context-free methods in reducing gradient variance. A key observation is that variance reduction becomes increasingly impactful as the model nears convergence. In the early stages of training, gradients are typically large and non-linear, but in the later stages, sampling noise becomes the dominant factor.
As illustrated in the results above, the variance reduction ratio (the variance of the new estimator divided by the variance of the standard estimator) drops significantly as the recognition network is trained, particularly in the "Late" stage of optimization. Furthermore, the method led to lower Negative Evidence Lower Bound (NELBO) values, indicating a superior fit to the data.
Discussion and Conclusion
Amortized variance reduction offers a powerful way to improve the efficiency of doubly stochastic optimization. Because control variates only add zero-expectation terms, the recognition network does not risk overfitting the main model; it simply accelerates the search for the optimum. While the recognition network introduces a small computational overhead (approximately 25% on CPU), the gains in convergence speed and stability make it a worthwhile investment for high-variance problems.
Future work may explore more flexible control variate forms to handle the complexities of models like Deep Gaussian Processes more robustly. Additionally, combining this approach with mini-batch variance reduction techniques like SVRG could provide even greater optimization stability.
Summary updated