Improving the particle filter for high-dimensional problems using artificial process noise

Anna Wigren, Lawrence M. Murray, Fredrik Lindsten

Abstract

The particle filter is one of the most successful methods for state inference and identification of general non-linear and non-Gaussian models. However, standard particle filters suffer from degeneracy of the particle weights for high-dimensional problems. We propose a method for improving the performance of the particle filter for certain challenging state space models, with implications for high-dimensional inference. First we approximate the model by adding artificial process noise in an additional state update, then we design a proposal that combines the standard and the locally optimal proposal. This results in a bias-variance trade-off, where adding more noise reduces the variance of the estimate but increases the model bias. The performance of the proposed method is evaluated on a linear Gaussian state space model and on the non-linear Lorenz'96 model. For both models we observe a significant improvement in performance over the standard particle filter.

Introduction

Particle filters (PFs) are essential tools for state estimation in complex, non-linear systems. Despite their versatility, they face a significant hurdle in high-dimensional applications: the Curse of Dimensionality. In these vast state spaces, the weights of the particles often collapse onto a single individual, a phenomenon known as weight degeneracy. To maintain a diverse set of particles, the required sample size typically grows exponentially with the system's dimension, rendering standard PFs computationally intractable for many real-world problems.

Methodology: Conjugate Artificial Process Noise

This paper introduces a novel approach to mitigate degeneracy by introducing Conjugate Artificial Process Noise. The core idea is to modify the underlying state-space model to allow for a more efficient sampling strategy. The method involves three key components:

  1. Model Approximation: The original model is augmented with an additional state update that injects artificial Gaussian noise, scaled by a parameter $\epsilon$.
  2. Locally Optimal Proposal: By choosing the artificial noise to be conjugate to the observation likelihood (e.g., Gaussian noise for Gaussian observations), the authors derive a "locally optimal" proposal. This allows particles to be analytically shifted toward current observations.
  3. Bias-Variance Trade-off: The parameter $\epsilon$ serves as a control knob. A low $\epsilon$ maintains high model fidelity (low bias) but risks weight degeneracy (high variance). Conversely, a higher $\epsilon$ stabilizes the filter (low variance) at the cost of introducing artificial noise into the system (higher bias).
Performance of the proposed method on a linear-Gaussian model across different noise levels ($\epsilon$). The plots show log-marginal likelihood ($\log Z$) and Mean Squared Error (MSE). Note the bias-variance tradeoff: as $\epsilon$ increases, variance decreases but bias eventually increases.

Experimental Results

The proposed method was rigorously tested on two benchmarks: a 10-dimensional linear-Gaussian model and the chaotic 10-dimensional Lorenz’96 model.

Performance and Accuracy

In the linear-Gaussian case, the artificial noise PF nearly matched the performance of the optimal Kalman Filter, whereas the standard bootstrap PF failed due to immediate degeneracy. For the Lorenz’96 model, which simulates complex atmospheric dynamics, the method showed significant improvements in both Mean Squared Error (MSE) and marginal log-likelihood estimates compared to standard baselines.

Robustness to Degeneracy

A critical finding is the relationship between $\epsilon$ and the Effective Sample Size (ESS). As $\epsilon$ increases, the probability of the filter collapsing (defined as $N_{eff} < 2$) drops sharply. This demonstrates that even a small amount of artificial noise can keep the particle population "healthy" and representative of the true posterior distribution.

Probability of particle filter degeneracy ($N_{eff} < 2$) as a function of artificial noise $\epsilon$ for the linear-Gaussian (left) and Lorenz'96 (right) models.

Conclusion

By intentionally introducing a controlled amount of model bias through artificial process noise, we can significantly improve the variance and stability of particle filters in high-dimensional settings. While the current approach relies on specific conjugacy pairs, it provides a robust framework for scaling PFs to more complex systems. Future work will focus on automated tuning of the noise parameters and extending the method to more diverse observation models.

Summary updated

Abstract

The particle filter is one of the most successful methods for state inference and identification of general non-linear and non-Gaussian models. However, standard particle filters suffer from degeneracy of the particle weights for high-dimensional problems. We propose a method for improving the performance of the particle filter for certain challenging state space models, with implications for high-dimensional inference. First we approximate the model by adding artificial process noise in an additional state update, then we design a proposal that combines the standard and the locally optimal proposal. This results in a bias-variance trade-off, where adding more noise reduces the variance of the estimate but increases the model bias. The performance of the proposed method is evaluated on a linear Gaussian state space model and on the non-linear Lorenz'96 model. For both models we observe a significant improvement in performance over the standard particle filter.
BibTeX
@inproceedings{wigren2018improving,
  author = {Anna Wigren and Lawrence M. Murray and Fredrik Lindsten},
  title = {{Improving the particle filter for high-dimensional problems using artificial process noise}},
  booktitle = {IFAC-PapersOnLine},
  year = {2018},
  doi = {10.1016/j.ifacol.2018.09.207},
}
Figure