An Iterative Nonlinear Filter Using Variational Bayesian Optimization

Yumei Hu, Xuezhi Wang, Hua Lan, Zengfu Wang, Bill Moran, Quan Pan2018 · DOI

TL;DR

This paper introduces a novel iterative nonlinear filter that leverages variational Bayesian optimization to improve state estimation accuracy in complex, non-linear systems. By iteratively refining a probabilistic approximation of the system's state, the proposed filter achieves better performance than existing methods like EKF and UKF, particularly in challenging tracking scenarios.

What Problem Does This Paper Address?

Many real-world systems, such as target tracking, autonomous navigation, and machine learning, involve nonlinear dynamics and measurements. Traditional Kalman filters, which provide optimal solutions for linear Gaussian systems, struggle with these nonlinearities, leading to intractable posterior probability density functions (PDFs). While various suboptimal methods exist, such as the Extended Kalman Filter (EKF) and Particle Filters (PF), they often involve approximations that can limit accuracy or computational efficiency. This paper aims to bridge this gap by developing an iterative nonlinear filter that offers improved accuracy by more effectively approximating the true posterior PDF through variational Bayesian techniques.

Methods

The core of this paper is the development of a Proximal Iterative Nonlinear Filter using Variational Bayes (PEKF-VB). The approach is rooted in variational Bayesian (VB) inference, which approximates a complex posterior PDF with a simpler, parameterized variational distribution. The key innovation lies in how this approximation is iteratively refined:

  • Evidence Lower Bound (ELBO) Maximization: The paper formulates the nonlinear estimation problem as maximizing the ELBO, which is a lower bound on the marginal likelihood of the data. This converts the intractable integration problem into a solvable optimization problem.
  • Kullback-Leibler (KL) Divergence as a Metric: The KL divergence is used to measure the difference between the true posterior PDF and the variational distribution. Minimizing this divergence is equivalent to maximizing the ELBO.
  • Proximal Iterative Scheme: To ensure convergence and a tight bound, a proximal point algorithm is employed. This algorithm iteratively updates the parameters of the variational distribution using a weighted KL divergence criterion, incorporating a penalty factor to balance accuracy and computational cost.
  • Closed-Form Solution: Through linearization and careful derivation, the iterative update steps for the state and its covariance are expressed in a closed-form, making the algorithm computationally feasible.
  • Core-Filter Initialization: The iterative process is initialized using a standard Bayesian filter (specifically, the EKF) to provide an initial estimate for the state and covariance.

Datasets

Not discussed in the paper. The paper uses simulated target tracking examples and a benchmark nonlinear filtering problem for evaluation.

Key Insights and Ideas

  • Iterative Refinement for Accuracy: The paper demonstrates that iteratively refining the variational approximation of the posterior PDF, rather than relying on a single-step approximation, significantly improves estimation accuracy in nonlinear systems.
  • VB as a Unifying Framework: Variational Bayes offers a principled way to handle complex probabilistic models, allowing for the derivation of iterative algorithms that can adapt to the specific nonlinearities of the system.
  • Trade-off Between Accuracy and Computation: The introduction of a penalty factor in the iterative update allows for tuning the algorithm's behavior, balancing the desire for tighter approximations with computational constraints.
  • Closed-Form Updates are Achievable: Despite the inherent complexity of nonlinear filtering, the authors show that by leveraging VB and specific mathematical techniques, closed-form iterative updates can be derived, making the method practical.

Key Results

The PEKF-VB algorithm was evaluated against EKF, UKF, and a related method called CODEKF (which uses Calvo and Oller's distance instead of KL divergence) in several simulated scenarios:

  • Range-Bearing Tracking: PEKF-VB achieved a slightly better Root Mean Squared Error (RMSE) than CODEKF, and both outperformed EKF and UKF.
  • Bearing-Only Tracking: PEKF-VB demonstrated the best target observability and a superior RMSE performance compared to EKF and UKF. While CODEKF also showed improvement, PEKF-VB was slightly better. The paper notes that the performance of both PEKF-VB and CODEKF improves significantly as more measurements accumulate.
  • Strongly Nonlinear Filtering Problem: PEKF-VB and CODEKF produced very similar and superior results compared to EKF and UKF, highlighting the advantage of iterative VB approaches in highly nonlinear scenarios.
  • Convergence: The ELBO consistently increased with each iteration, indicating that the PEKF-VB algorithm converges. The KL divergence also decreased towards zero, confirming the effectiveness of the variational approximation.

Limitations and Future Work

  • Computational Overhead: The paper acknowledges that the iterative nature of PEKF-VB, while improving accuracy, comes at the cost of increased computational burden compared to single-step filters like EKF and UKF.
  • Gaussian Assumption: The derivation of closed-form solutions relies on Gaussian assumptions for process and measurement noise. While the VB framework can be extended to non-Gaussian distributions using Monte Carlo methods, this would increase computational complexity.
  • Future Work: The authors suggest exploring other divergence measures beyond KL divergence and applying the VB framework to non-Gaussian distributions. Further investigation into optimizing the penalty factor and exploring more complex nonlinear systems are also implied.

Summary updated

BibTeX
@misc{hu2018iterative,
  author = {Yumei Hu and Xuezhi Wang and Hua Lan and Zengfu Wang and Bill Moran and Quan Pan},
  title = {{An Iterative Nonlinear Filter Using Variational Bayesian Optimization}},
  year = {2018},
  doi = {10.3390/s18124222},
}
Figure