Variational State and Parameter Estimation

Jarrad Courts, Johannes N. Hendriks, Adrian Wills, Thomas B. Schön, Brett NinnessIFAC-PapersOnLine 2020 · DOI · OpenReview

TL;DR

This paper introduces a novel variational inference (VI) approach for estimating both the hidden states and unknown parameters of nonlinear state-space models. By formulating this as an optimization problem with readily available derivatives, the method offers a computationally efficient alternative to traditional methods like Hamiltonian Monte Carlo (HMC), achieving comparable results on simulated and real-world data.

What Problem Does This Paper Address?

Estimating the hidden states and parameters of nonlinear state-space models is a fundamental problem in many scientific and engineering fields. These models are described by a sequence of latent states that evolve over time and are observed through noisy measurements. The challenge lies in the fact that obtaining exact Bayesian estimates (i.e., the posterior distribution) for both states and parameters is generally intractable due to the nonlinearities and the complex integrals involved. This necessitates the use of approximation techniques. While methods like Hamiltonian Monte Carlo (HMC) can provide accurate estimates, they are often computationally expensive. This paper aims to bridge this gap by developing a deterministic, computationally efficient variational inference method that approximates the desired posterior distribution.

Methods

The core of this paper is the application of Variational Inference (VI) to approximate the intractable posterior distribution $p(x, \theta | y)$, where $x$ represents the states and $\theta$ represents the model parameters, given measurements $y$. VI works by introducing a tractable, parameterized approximating distribution, $q_\beta(x, \theta)$, and then finding the parameters $\beta$ that minimize the Kullback-Leibler (KL) divergence between $q_\beta$ and the true posterior. This minimization is equivalent to maximizing a lower bound on the log-likelihood of the data, denoted as $L(\beta)$.

The authors propose a specific parametric form for $q_\beta$: a multivariate Gaussian distribution for the joint state-parameter sequence at each time step. This choice is crucial because it allows for the efficient computation of the gradients (first and second-order derivatives) of the lower bound $L(\beta)$, which are necessary for optimization. The paper details how the complex integral for $L(\beta)$ can be decomposed into terms that depend on pairwise joint distributions of states and parameters at consecutive time steps, making the computation tractable even for long time series. This decomposition, combined with the Gaussian assumption, allows for the use of standard optimization techniques, such as gradient-based methods, to find the optimal parameters $\beta$.

Datasets

The paper evaluates its proposed method on two numerical examples:

  1. Stochastic Volatility Model: This is a simulated dataset with 726 measurements, used to estimate parameters of a nonlinear model.
  2. Inverted Pendulum (Furata Pendulum): This is a real-world dataset collected from a Quanser QUBE-Servo 2, consisting of 375 measurements. This example involves estimating a four-dimensional state vector and six parameters, along with a covariance matrix.

The paper does not introduce new benchmark datasets but uses these examples to compare its method against state-of-the-art Hamiltonian Monte Carlo (HMC) implementations.

Key Insights and Ideas

  • Tractability through Variational Inference: The paper demonstrates that variational inference can transform a generally intractable Bayesian estimation problem into a tractable optimization problem.
  • Gaussian Assumption for Efficiency: By assuming a Gaussian form for the approximating distribution, the authors achieve analytical tractability for derivatives, enabling efficient gradient-based optimization. This is a key trade-off: while it limits the ability to capture highly non-Gaussian posterior shapes, it significantly boosts computational speed.
  • Decomposition for Scalability: The clever decomposition of the objective function $L(\beta)$ into terms involving only pairwise joint distributions of states and parameters at consecutive time steps is crucial for scaling the method to long time series.
  • Deterministic vs. Stochastic Approximation: The proposed method is deterministic, providing a single point estimate for the parameters of the approximating distribution, unlike stochastic methods like HMC which explore the posterior distribution. This can be advantageous for applications requiring speed and a single best estimate.
  • Overconfidence in Non-Gaussian Cases: The paper acknowledges that the Gaussian assumption can lead to overconfident estimates when the true posterior is significantly non-Gaussian, as observed in the stochastic volatility example for the parameter $\log(c)$.

Key Results

  • Computational Efficiency: The proposed variational inference method is significantly faster than HMC. For the stochastic volatility example, it required 17.6 seconds compared to HMC's 87.0 seconds. For the inverted pendulum, the proposed method took "a few minutes" while HMC required "a few hours."
  • Comparable Accuracy: Despite the computational advantage, the proposed method achieves results that are comparable to HMC. In the stochastic volatility example, state distributions closely matched HMC. In the inverted pendulum example, the parameter distributions were also close, although the VI method was noted to be overconfident in some cases.
  • Flexibility: The method can handle estimation of both states and parameters simultaneously and can even accommodate point estimates for some parameters (like the covariance matrix in the inverted pendulum example) while treating others as distributions.

Limitations and Future Work

  • Gaussian Assumption Limitation: The primary limitation acknowledged is the assumption of a Gaussian approximating distribution. This can lead to overconfident posteriors when the true posterior is non-Gaussian, as seen with the $\log(c)$ parameter in the stochastic volatility example.
  • Local Optima: The optimization problem might converge to a local optimum rather than the global optimum, especially for complex models.
  • Future Work: The authors suggest blending their approach with Sequential Monte Carlo (SMC) algorithms. This would aim to combine the computational efficiency of their variational method with the theoretical guarantees of SMC, potentially overcoming the limitations imposed by the Gaussian assumption and exploring more complex posterior distributions.

Summary updated

Abstract

This paper considers the problem of computing Bayesian estimates of both states and model parameters for nonlinear state-space models. Generally, this problem does not have a tractable solution and approximations must be utilised. In this work, a variational approach is used to provide an assumed density which approximates the desired, intractable, distribution. The approach is deterministic and results in an optimisation problem of a standard form. Due to the parametrisation of the assumed density selected first- and second-order derivatives are readily available which allows for efficient solutions. The proposed method is compared against state-of-the-art Hamiltonian Monte Carlo in two numerical examples.
BibTeX
@inproceedings{courts2020variational,
  author = {Jarrad Courts and Johannes N. Hendriks and Adrian Wills and Thomas B. Schön and Brett Ninness},
  title = {{Variational State and Parameter Estimation}},
  booktitle = {IFAC-PapersOnLine},
  year = {2020},
  doi = {10.1016/j.ifacol.2021.08.448},
}
Figure