Abstract
Many recent advances in large scale probabilistic inference rely on variational methods. The success of variational approaches depends on (i) formulating a flexible parametric family of distributions, and (ii) optimizing the parameters to find the member of this family that most closely approximates the exact posterior. In this paper we present a new approximating family of distributions, the variational sequential Monte Carlo (VSMC) family, and show how to optimize it in variational inference. VSMC melds variational inference (VI) and sequential Monte Carlo (SMC), providing practitioners with flexible, accurate, and powerful Bayesian inference. The VSMC family is a variational family that can approximate the posterior arbitrarily well, while still allowing for efficient optimization of its parameters. We demonstrate its utility on state space models, stochastic volatility models for financial data, and deep Markov models of brain neural circuits.
Introduction
Probabilistic machine learning often centers on the challenge of posterior inference—updating beliefs about latent variables given observed data. Traditionally, researchers have faced a trade-off between the scalability of Variational Inference (VI) and the flexibility of Sequential Monte Carlo (SMC). While VI is efficient, it is often constrained by the limited expressivity of the chosen variational family. Conversely, SMC can approximate complex distributions with high fidelity using "particles," but it lacks a natural optimization framework for tuning its parameters.
Variational Sequential Monte Carlo (VSMC) bridges this gap by introducing a new variational family based on the SMC process. This framework allows for the optimization of SMC proposals using variational techniques, combining the best of both worlds.
The Variational Sequential Monte Carlo Family
The VSMC family is defined by the distribution of a trajectory sampled from an SMC algorithm. By running an SMC process with $N$ particles and selecting one based on its importance weight, we obtain a flexible approximation of the posterior. A notable theoretical insight is that the Importance Weighted Auto-Encoder (IWAE) is a special case of VSMC where the resampling step is omitted.
To optimize the parameters of this family, the authors derive a surrogate Evidence Lower Bound (ELBO) based on the SMC estimate of the marginal likelihood. This surrogate bound is maximized using stochastic gradient ascent, allowing the model to "learn" the optimal proposal distribution.
The Role of Resampling in Temporal Stability
A critical advantage of VSMC over simple importance sampling methods like IWAE is its use of resampling. In long time-series models, importance sampling often suffers from particle collapse, where the effective sample size drops to one, leading to poor approximations. VSMC’s resampling mechanism maintains a diverse set of particles, ensuring stability and accuracy even as the sequence length $T$ increases.
Optimization and Gradient Variance Reduction
Optimizing the VSMC objective presents challenges due to the discrete nature of resampling. The standard "score function" gradient for these discrete choices typically exhibits high variance, hindering convergence. The authors found that a "biased gradient" approach—omitting the score function terms and focusing on the reparameterized gradients of the proposal—acts as a highly effective variance reduction technique. This allows for stable training of complex models, including deep neural architectures, with almost no loss in final accuracy.
Experimental Evaluation
The authors evaluated VSMC across several challenging domains:
- Linear Gaussian State Space Models: VSMC achieved an ELBO within 0.9 nats of the true log-marginal likelihood, significantly outperforming standard bootstrap filters.
- Stochastic Volatility: In financial time-series analysis of exchange rates, VSMC (with 16 particles) achieved an ELBO of 6936.6, surpassing both Structured VI (6905.1) and IWAE (6913.3).
- Deep Markov Models: On motor cortex neural recording data, VSMC demonstrated superior training stability and reached high accuracy levels more efficiently than competing methods.
Conclusion
Variational Sequential Monte Carlo provides a powerful framework for inference in complex dynamical systems. By combining the optimization strengths of VI with the asymptotic consistency of SMC, it offers a scalable path toward high-fidelity Bayesian inference. While it introduces additional computational and memory costs proportional to the number of particles, the gains in accuracy and stability make it a compelling choice for demanding time-series applications.
Summary updated