Kalman Filtering Under Information Theoretic Criteria

Badong Chen, Lujuan Dang, Nanning Zheng, José C. Prı́ncipe, José C. PríncipeOpenalex Unknown 2023 · DOI · OpenAlex

Abstract

The Kalman filter is a fundamental tool in state estimation, yet its performance is intrinsically tied to the assumption of Gaussian noise through the Minimum Mean Square Error (MMSE) criterion. In practical engineering scenarios, noise is often non-Gaussian, featuring outliers, skewness, or multimodal characteristics. This paper discusses the transition from MMSE to Information Theoretic Criteria (ITC) for Kalman filtering. By leveraging Information Theoretic Learning (ITL), specifically the Maximum Correntropy Criterion (MCC) and Minimum Error Entropy (MEE), these filters achieve enhanced robustness without the prohibitive computational cost of non-parametric methods like Particle Filters.

Introduction

For decades, the Kalman filter has served as the "gold standard" for state estimation in navigation, tracking, and control systems. Its mathematical elegance stems from the MMSE criterion, which is optimal for linear systems under Gaussian noise. However, sensors in the real world frequently encounter "glitches" or impulsive noise that deviate from the bell-shaped Gaussian curve. When such outliers occur, the MMSE criterion—which focuses on the second-order statistic (variance)—over-adjusts to minimize the large error, leading to instability or divergence.

The Challenge of Non-Gaussian Noise

Traditional approaches to non-Gaussian estimation often involve Particle Filters (PF). While PFs can approximate any distribution, they require a massive number of samples, making them computationally expensive and difficult to tune for real-time embedded systems. There is a clear need for a middle ground: a filter that retains the recursive efficiency of the Kalman framework while gaining the robustness of information-theoretic approaches.

Methodology: Shifting to Information Theoretic Criteria

The core innovation presented is the replacement of the MMSE cost function with criteria derived from Information Theoretic Learning (ITL).

Maximum Correntropy Criterion (MCC)

MCC acts as a localized similarity measure. Unlike the squared error, which grows quadratically, the correntropy function is bounded. This property allows the filter to effectively "ignore" large outliers that fall outside the primary distribution, making the Maximum Correntropy Kalman Filter (MCKF) significantly more stable in impulsive noise environments.

Minimum Error Entropy (MEE)

While MMSE minimizes the spread of the error, MEE minimizes the uncertainty (entropy) of the error distribution. This approach allows the filter to account for higher-order statistics, making it particularly effective for skewed or multimodal noise. To ensure the filter remains unbiased, the authors utilize "Minimum Error Entropy with Fiducial Points" (MEEF), anchoring the error distribution to zero.

Integration with Standard Architectures

These criteria are not standalone algorithms but are integrated into existing Kalman variants. This results in a robust suite of tools:

  • Extended Kalman Filter (EKF) with ITC
  • Unscented Kalman Filter (UKF) with ITC
  • Cubature Kalman Filter (CKF) with ITC

Key Insights and Practicality

A significant advantage of ITC-based filters is their computational profile. By maintaining a recursive structure, they avoid the "curse of dimensionality" associated with particle-based methods. Furthermore, the exploration of Reproducing Kernel Hilbert Spaces (RKHS) provides a theoretical bridge between classical control theory and modern machine learning, allowing for more sophisticated kernel-based estimation.

Validation through illustrative examples, including power system state estimation, demonstrates that these filters provide superior robustness. They serve as "plug-and-play" replacements for standard filters in existing software pipelines, requiring only "reasonable computation" overhead.

Limitations and Considerations

Despite their advantages, ITC-based filters introduce new challenges:

  1. Parameter Selection: Criteria like MCC require the tuning of a kernel bandwidth parameter. Optimal selection is critical for performance.
  2. Convergence: Because ITC objective functions can be non-convex, ensuring stability and convergence requires careful algorithmic design.
  3. Complexity Trade-offs: While lighter than Particle Filters, these methods are more complex than the basic Kalman filter, necessitating a cost-benefit analysis for extremely low-power applications.

Conclusion

Kalman filtering under Information Theoretic Criteria represents a significant step forward in robust state estimation. By moving beyond second-order statistics, these filters provide the resilience needed for real-world, non-Gaussian environments while remaining practical for real-time engineering applications.

Summary updated

Abstract

This book provides efficient Kalman filters under information theoretic criteria, achieving performance in non-Gaussian noises with non-expensive computation.
BibTeX
@inproceedings{chen2023kalman,
  author = {Badong Chen and Lujuan Dang and Nanning Zheng and José C. Prı́ncipe and José C. Príncipe},
  title = {{Kalman Filtering Under Information Theoretic Criteria}},
  booktitle = {Openalex Unknown},
  year = {2023},
  doi = {10.1007/978-3-031-33764-2},
}
Figure