Variational Sequential Monte Carlo

Christian A. Naesseth, Scott W. Linderman, Rajesh Ranganath, David M. Blei

Abstract

Many recent advances in large scale probabilistic inference rely on variational methods. The success of variational approaches depends on formulating a flexible parametric family of distributions and efficiently optimizing its parameters. Variational Sequential Monte Carlo (VSMC) introduces a powerful hybrid framework that combines the optimization-driven speed of Variational Inference (VI) with the flexible, multi-sample accuracy of Sequential Monte Carlo (SMC). By treating the SMC process itself as a variational family, VSMC can approximate complex posteriors arbitrarily well by increasing the number of particles, effectively solving the "particle degeneracy" problem that plagues previous methods like Importance Weighted Auto-Encoders (IWAE) on long sequences.

Introduction

Probabilistic modeling often requires estimating a "posterior"—the probability of hidden states given noisy observations. Traditionally, researchers face a trade-off: Variational Inference (VI) is fast and turns inference into an optimization problem but is often biased by simple distribution families (e.g., Gaussians). Conversely, Sequential Monte Carlo (SMC) is highly flexible and accurate but depends heavily on a well-designed "proposal distribution."

Existing multi-sample variational methods, such as the Importance Weighted Auto-Encoder (IWAE), struggle with long time-series data. As time progresses, IWAE suffers from "particle collapse," where a single sample carries all the weight, rendering the multi-sample approach ineffective. VSMC bridges this gap by incorporating SMC's resampling mechanism directly into a variational framework.

The VSMC Framework

The core innovation of VSMC is the definition of a variational family based on an entire SMC sampling run. Instead of picking a static distribution, the authors use the output of SMC as the approximating distribution.

Resampling and Temporal Stability

Unlike IWAE, which propagates particles independently, VSMC utilizes resampling at each time step. This process selects high-weight particles and discards low-weight ones, maintaining a diverse set of samples even over long sequences. This prevents the "particle collapse" observed in non-resampling methods.

Comparison of VSMC and IWAE dynamics. (a) VSMC maintains diversity through resampling. (b) IWAE suffers from particle collapse. (c) VSMC maintains a tighter ELBO over time compared to IWAE and standard VB.

Optimization and the "Gradient Hack"

To optimize the proposal distributions, the authors derive a "Surrogate ELBO" based on the SMC marginal likelihood estimate. They prove this is a valid lower bound on the true log-marginal likelihood. A major practical contribution is the discovery that the gradient of the objective can be simplified. By ignoring the high-variance "score function" term related to discrete resampling and focusing on the reparameterization term, the authors achieved faster convergence and stable training with nearly identical final accuracy.

Empirical evidence for the 'gradient hack': the reparameterization term (blue) closely tracks the full gradient (red) while exhibiting significantly lower variance, making it a superior choice for optimization.

Experimental Evaluation

The authors evaluated VSMC across three distinct models:

  • Linear Gaussian State Space Model (SSM): A synthetic benchmark where the true posterior can be calculated exactly.
  • Stochastic Volatility Model: Applied to 10 years of monthly exchange rate returns. VSMC achieved a significantly tighter ELBO (6936.6) compared to IWAE (6913.3) and standard Structured VI (6905.1).
  • Deep Markov Model: High-dimensional, non-linear dynamics of neural recordings from a macaque motor cortex. VSMC reached higher ELBO values faster than IWAE and scaled effectively with the number of particles ($N$).

Key Insights

  • Fidelity vs. Complexity: VSMC provides a "knob" for accuracy. Unlike standard VI, VSMC becomes arbitrarily accurate as the number of particles increases.
  • Learning Optimal Proposals: VSMC allows the model to learn high-quality proposal distributions through gradient descent, which is often mathematically impossible to derive manually in complex models.
  • The Power of Biased Gradients: The ability to ignore discrete ancestor gradients simplifies implementation and improves stability, a significant win for practitioners.

Limitations and Future Work

While powerful, VSMC incurs higher computational overhead due to running SMC inside an optimization loop. Current formulations also focus on continuous latent variables; extending this to discrete state spaces would require alternative gradient estimators. Future work will also explore the theoretical conditions under which the "biased" gradient remains robust.

Summary updated

Abstract

Many recent advances in large scale probabilistic inference rely on variational methods. The success of variational approaches depends on (i) formulating a flexible parametric family of distributio...
BibTeX
@inproceedings{naesseth2018variational,
  author = {Christian A. Naesseth and Scott W. Linderman and Rajesh Ranganath and David M. Blei},
  title = {{Variational Sequential Monte Carlo}},
  booktitle = {International Conference on Artificial Intelligence and Statistics},
  year = {2018},
  url = {https://openreview.net/forum?id=AXmxdb_jdH},
}
Figure