VISA: Variational Inference with Sequential Sample-Average Approximations

Heiko Zimmermann, Christian A. Naesseth, Jan-Willem van de Meent

Introduction

In fields like ecology, climate science, and physics, researchers frequently rely on complex, computationally expensive simulators. These models are often "non-differentiable"—meaning standard gradient-based optimization tricks (like the reparameterization trick used in VAEs) cannot be applied due to discrete logic or stochastic control flow within the simulator.

To perform approximate inference on these "black-box" models, the standard approach is Importance-Weighted Forward-KL Variational Inference (IWFVI). However, IWFVI suffers from a massive computational bottleneck: it requires a fresh set of samples from the expensive simulator at every single step of the optimization process. If a simulator takes minutes to run and the optimizer requires thousands of steps to converge, the total compute time quickly becomes prohibitive.

VISA (Variational Inference with Sequential Sample-Average Approximations) is a novel method designed to solve this exact problem. By intelligently reusing model evaluations across multiple gradient steps, VISA achieves the same accuracy as standard methods but at a fraction of the computational cost.

The VISA Approach

VISA identifies a missed opportunity in standard IWFVI: if the model parameters only change slightly between optimization steps, the expensive simulations we just ran should still be highly relevant. VISA capitalizes on this by combining IWFVI with Sample-Average Approximations (SAA).

Algorithm and trust region diagram for VISA.

Here is how the VISA algorithm operates:

  1. The Surrogate Objective: Instead of resampling at every step, VISA fixes a set of samples and treats the resulting importance-weighted objective as a deterministic "surrogate" function. This allows the optimizer to take multiple gradient steps without calling the expensive simulator.
  2. The Trust Region: Samples cannot be reused indefinitely. As the variational distribution ($q_\phi$) evolves during optimization, the old samples become less representative of the current state. VISA establishes a "trust region" within which the current samples are deemed valid.
  3. ESS Gatekeeping: To determine the boundaries of this trust region, VISA monitors the Effective Sample Size (ESS). The ESS measures how well the current distribution matches the one used to generate the original samples. When the ESS drops below a predefined threshold ($\alpha$), it triggers a "refresh": the algorithm draws new samples, runs the simulator, and constructs a new SAA.
  4. Gradient-Free Simulators: Unlike previous SAA methods for variational inference, VISA does not require the underlying model to be differentiable, making it a versatile drop-in replacement for standard IWFVI.

Experimental Setup

The researchers evaluated VISA across three distinct problem settings:

  • High-Dimensional Gaussians: Synthetic targets ($D=128$ for diagonal covariance and $D=32$ for dense covariance) used to calculate exact approximation errors.
  • Lotka-Volterra Dynamics: A classic biological predator-prey ODE system, testing the model's ability to infer population parameters and initial conditions.
  • Pickover Attractor: A chaotic 3D system requiring a compute-intensive bootstrap particle filter, serving as a "hard mode" test for the algorithm.

Key Results and Insights

VISA is particularly well-suited for scenarios exhibiting an asymmetry in compute costs—specifically, where evaluating the variational posterior is cheap, but evaluating the model likelihood is highly expensive.

Convergence plots comparing VISA to baselines across different learning rates.
  • Significant Speedups: VISA demonstrated massive efficiency gains, particularly when using conservative (smaller) learning rates. While standard IWFVI wastes enormous amounts of compute on tiny updates in these "slow lanes," VISA simply reuses its simulations to cruise through these steps, reaching convergence with significantly fewer model evaluations.
  • Comparable Accuracy: Across all experiments, VISA matched the final approximation quality of standard IWFVI.
  • Deterministic Stability: Because the surrogate objective remains deterministic between refreshes, VISA exhibits more stable optimization trajectories compared to the high-variance "chatter" typical of standard stochastic gradient methods. This proved especially robust on chaotic systems like the Pickover Attractor.
  • Optimal Thresholds: Empirical results suggest that an ESS threshold ($\alpha$) between 0.3 and 0.6 provides the best balance between computational speed and inference accuracy.

Limitations and Future Work

While powerful, VISA does have limitations. Like all importance-sampling methods, it is susceptible to the curse of dimensionality. In extremely high-dimensional spaces, the ESS drops too rapidly, forcing frequent refreshes and negating the computational savings.

Additionally, there is a risk of overfitting if the trust region is set too large (i.e., the $\alpha$ threshold is too low). The optimizer may overfit to a few high-weight samples, leading to an underestimation of the posterior variance. Finally, while the authors experimented with second-order optimizers like L-BFGS, they found it could sometimes amplify this overfitting behavior. Future work will likely focus on refining how second-order methods interact with sequential approximations to further enhance speed and stability.

Summary updated

Abstract

We present variational inference with sequential sample-average approximations (VISA), a method for approximate inference in computationally intensive models, such as those based on numerical simulations. VISA extends importance-weighted forward-KL variational inference by employing a sequence of sample-average approximations, which are considered valid inside a trust region. This makes it possible to reuse model evaluations across multiple gradient steps, thereby reducing computational cost. We perform experiments on high-dimensional Gaussians, Lotka-Volterra dynamics, and a Pickover attractor, which demonstrate that VISA can achieve comparable approximation accuracy to standard importance-weighted forward-KL variational inference with computational savings of a factor two or more for conservatively chosen learning rates.
BibTeX
@inproceedings{zimmermann2024visa,
  author = {Heiko Zimmermann and Christian A. Naesseth and Jan-Willem van de Meent},
  title = {{VISA: Variational Inference with Sequential Sample-Average Approximations}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2024},
  url = {https://openreview.net/forum?id=lbLC5OV9GY},
}
Figure