Efficient Bayesian inference for stochastic volatility models with ensemble MCMC methods

Alexander Y. Shestopaloff, Radford M. NealarXiv 2014 · DOI · arXiv

TL;DR

This paper introduces new ensemble Markov Chain Monte Carlo (MCMC) methods that speed up Bayesian inference for stochastic volatility models by a factor of 3.1 compared to previous state-of-the-art techniques. Unlike existing methods that rely on linear Gaussian approximations, these ensemble samplers work with the exact model density, making them both more efficient and more flexible for complex financial modeling.

What Problem Does This Paper Address?

Stochastic Volatility (SV) models are the bread and butter of financial econometrics, used to model time-varying risk in stock returns. However, they are notoriously difficult to estimate. They are "state-space models" where the thing we observe (stock returns) has a non-linear and non-Gaussian relationship with the thing we want to track (the hidden "volatility" state).

Because the relationship isn't linear or Gaussian, standard tools like the Kalman filter don't work. Previous "state-of-the-art" samplers (like the one by Kastner and Fruwirth-Schnatter) get around this by using a 10-component Gaussian mixture to approximate the model. While clever, this approximation adds complexity and limits the model's generalizability. The authors sought a way to perform inference that is both faster and more "exact."

Methods

The authors leverage Ensemble MCMC, a framework where the sampler doesn't just look at one possible state at a time, but a collection (an "ensemble") of states simultaneously.

The "secret sauce" of this paper is how they make this computationally feasible:

  1. The Forward Algorithm & Caching: Evaluating an ensemble of $K$ points usually takes $K$ times as long as one point. However, the authors show that by using the "forward algorithm" and a "caching" technique to store transition probabilities, they can evaluate the entire ensemble's density in nearly the same time it takes to evaluate a single point.
  2. ENS1 and ENS2: They propose two schemes. ENS1 maps the current state to an ensemble of latent sequences and parameters, then samples a new state from that ensemble. ENS2 goes a step further by performing a Metropolis update on the persistence parameter ($\phi$) while "integrated" over the ensemble, allowing for much larger, more effective moves in the parameter space.
  3. Sufficient Statistics: They derive low-dimensional sufficient statistics for the SV model, allowing them to perform multiple parameter updates (80+ per iteration) at almost zero extra computational cost.

Datasets

The methods were evaluated on a simulated financial time series generated from a standard stochastic volatility model:

  • Size: $N = 1000$ time points.
  • Parameters: $c = 0.5$ (constant), $\phi = 0.98$ (high persistence), and $\sigma^2 = 0.15$ (volatility of volatility).
  • Priors: Standard Bayesian priors (Normal for $c$, Uniform for $\phi$, and Inverse-Gamma for $\sigma^2$).

Key Insights and Ideas

  • Fast Variables vs. Slow Variables: The authors treat the log-variance ($\eta$) as a "fast" variable. Because they cache the transition probabilities of the latent states, they can test many different values of $\eta$ extremely quickly.
  • Breaking the Approximation Dependency: Most SV samplers are tied to the "Kim et al. (1998)" approximation. This paper proves you can achieve better performance by staying "exact," which opens the door for models with even weirder noise structures where Gaussian approximations would fail.
  • Ensembles Provide Global Context: By looking at a pool of possible latent sequences, the sampler is less likely to get "stuck" in a local region of the posterior distribution. It can propose a change to a parameter and find a latent sequence in the ensemble that "fits" that new parameter immediately.

Key Results

The primary metric used was Autocorrelation Time (ACT) adjusted for computation time—essentially, "how much useful information do we get per second of CPU time?"

  • 3.1x Speedup: The ENS1 method was roughly 3.1 times more efficient than the improved state-of-the-art (KF) sampler for the most difficult parameter to estimate ($\eta$).
  • Robustness: The ensemble methods were not overly sensitive to tuning parameters (like the size of the ensemble), provided the pools were large enough (e.g., 50 latent states and 10 parameter values).
  • ENS1 vs. ENS2: Interestingly, the simpler ENS1 outperformed the more complex ENS2 in this specific test, though ENS2 remains a theoretically powerful tool for even harder problems.

Limitations and Future Work

  • Tuning ENS2: The authors noted that ENS2 didn't provide the expected performance gain over ENS1, suggesting that the "ensemble Metropolis" update for $\phi$ needs better tuning or may be more beneficial in different model contexts.
  • Multivariate Scaling: While this paper focuses on univariate models (one stock), the authors mention that multivariate versions are the next frontier. In multivariate cases, Gaussian approximations are even harder to find, making ensemble methods an even more attractive candidate.
  • Comparison to PMCMC: The authors suggest future research comparing these ensemble methods to Particle MCMC (PMCMC) techniques to see which handles high-dimensional latent spaces better.

Summary updated

Abstract

In this paper, we introduce efficient ensemble Markov Chain Monte Carlo (MCMC) sampling methods for Bayesian computations in the univariate stochastic volatility model. We compare the performance of our ensemble MCMC methods with an improved version of a recent sampler of Kastner and Fruwirth-Schnatter (2014). We show that ensemble samplers are more efficient than this state of the art sampler by a factor of about 3.1, on a data set simulated from the stochastic volatility model. This performance gain is achieved without the ensemble MCMC sampler relying on the assumption that the latent process is linear and Gaussian, unlike the sampler of Kastner and Fruwirth-Schnatter.
BibTeX
@misc{shestopaloff2014efficient,
  author = {Alexander Y. Shestopaloff and Radford M. Neal},
  title = {{Efficient Bayesian inference for stochastic volatility models with ensemble MCMC methods}},
  year = {2014},
  eprint = {1412.3013},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1412.3013},
}
Figure