Nonlinear Bayesian Filtering with Natural Gradient Gaussian Approximation

Wenhan Cao, Tianyi Zhang, Zeju Sun, Chang Liu, Stephen S.-t. Yau, Shengbo Eben LiarXiv 2024 · DOI · arXiv

Abstract

Nonlinear Bayesian filtering is essential for state estimation in robotics and aerospace, yet exact solutions are often computationally intractable. Traditional Gaussian filters like the Extended Kalman Filter (EKF) and Unscented Kalman Filter (UKF) rely on local linearization or statistical regression, which can introduce substantial errors. We present the NANO filter (Natural grAdient GaussiaN apprOximation), which treats filtering as an optimization problem on the Gaussian manifold. By utilizing natural gradient descent (NGD) to account for the curvature of probability distributions, NANO achieves a 45% reduction in estimation error over traditional methods while remaining efficient for real-time applications.

Introduction

In complex systems like autonomous drones or air-traffic control, estimating the system "state" from noisy sensor data is a fundamental challenge. Bayesian filtering provides the theoretical framework for this estimation. However, for nonlinear systems, the true posterior distribution is rarely Gaussian. To maintain tractability, engineers typically use Gaussian filters that approximate the posterior with a bell curve.

The primary source of error in these filters is often the "enabling approximation"—the method used to linearize the nonlinear system model. As shown in Figure 1, techniques like Taylor Series Expansion (used in EKF) or Statistical Linear Regression (used in UKF) simplify the math before the update step, often failing to find the optimal Gaussian fit.

Comparison of traditional linearization techniques and their representative algorithms.

The NANO Filter Framework

The NANO filter shifts the paradigm from linearizing the model to directly optimizing the Gaussian parameters. It treats the prediction and update steps as variational problems solved via Natural Gradient Descent (NGD).

Natural Gradient on the Gaussian Manifold

Standard gradient descent assumes a flat Euclidean parameter space. In contrast, NGD uses the Fisher Information Matrix to perform steepest descent on the Riemannian manifold of Gaussian distributions. This geometric approach accounts for the natural curvature of probability space, leading to faster convergence and higher accuracy.

Algorithm and Implementation

The NANO filter is implemented as an iterative process within each time step. Leveraging Stein’s Lemma, the authors derive a derivative-free version that avoids the need for complex Jacobian or Hessian matrices, making it robust for highly nonlinear functions.

Algorithm 1: The NANO Filter pseudocode, detailing the iterative update process.

The framework is also extensible to "Gibbs posteriors," allowing for robust loss functions (e.g., Huber loss) that mitigate the impact of sensor outliers, such as GPS glitches.

Experimental Results

The NANO filter was evaluated across three distinct scenarios:

  1. Wiener Velocity Model: Confirmed that NANO reduces to the standard Kalman Filter in linear cases.
  2. Air-Traffic Control: In highly nonlinear maneuvering simulations, NANO reduced Root Mean Square Error (RMSE) by approximately 45% compared to EKF, UKF, and the Posterior Linearization Filter (PLF).
  3. Real-World UGV Localization: Using Lidar data from an Unmanned Ground Vehicle, NANO demonstrated superior tracking performance over 11-minute trajectories.

Despite being iterative, NANO is computationally efficient. Its average runtime is roughly 2-3 times that of the EKF (e.g., 2ms vs 1ms), which is well within the requirements for real-time robotics.

Conclusion

The NANO filter provides a rigorous geometric reinterpretation of Bayesian filtering. By replacing heuristic linearization with principled natural gradient optimization, it offers a significant leap in accuracy without sacrificing real-time feasibility. Future work will explore global convergence guarantees for multi-modal distributions and deeper Riemannian geometric properties of the Gaussian manifold.

Summary updated

Abstract

Practical Bayes filters often assume the state distribution of each time step to be Gaussian for computational tractability, resulting in the so-called Gaussian filters. When facing nonlinear systems, Gaussian filters such as extended Kalman filter (EKF) or unscented Kalman filter (UKF) typically rely on certain linearization techniques, which can introduce large estimation errors. To address this issue, this paper reconstructs the prediction and update steps of Gaussian filtering as solutions to two distinct optimization problems, whose optimal conditions are found to have analytical forms from Stein's lemma. It is observed that the stationary point for the prediction step requires calculating the first two moments of the prior distribution, which is equivalent to that step in existing moment-matching filters. In the update step, instead of linearizing the model to approximate the stationary points, we propose an iterative approach to directly minimize the update step's objective to avoid linearization errors. For the purpose of performing the steepest descent on the Gaussian manifold, we derive its natural gradient that leverages Fisher information matrix to adjust the gradient direction, accounting for the curvature of the parameter space. Combining this update step with moment matching in the prediction step, we introduce a new iterative filter for nonlinear systems called \textit{N}atural Gr\textit{a}dient Gaussia\textit{n} Appr\textit{o}ximation filter, or NANO filter for short. We prove that NANO filter locally converges to the optimal Gaussian approximation at each time step. Furthermore, the estimation error is proven exponentially bounded for nearly linear measurement equation and low noise levels through constructing a supermartingale-like property across consecutive time steps.
BibTeX
@misc{cao2024nonlinear,
  author = {Wenhan Cao and Tianyi Zhang and Zeju Sun and Chang Liu and Stephen S.-T. Yau and Shengbo Eben Li},
  title = {{Nonlinear Bayesian Filtering with Natural Gradient Gaussian Approximation}},
  year = {2024},
  eprint = {2410.15832},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2410.15832},
}
Figure