Latent-KalmanNet: Learned Kalman Filtering for Tracking From High-Dimensional Signals

Itay Buchnik, Guy Revach, Damiano Steger, Ruud J. G. van Sloun, Tirza Routtenberg, Nir ShlezingerIEEE TSP 2024 · DOI

TL;DR

This paper introduces Latent-KalmanNet, a novel deep learning framework that combines a learned encoder with a KalmanNet-based tracker to achieve accurate and efficient state estimation from high-dimensional, noisy, and potentially partially observable signals. It addresses limitations of traditional Kalman filters and existing data-driven methods by jointly learning a latent representation and a tracking model, leading to improved performance and robustness.

What Problem Does This Paper Address?

Tracking the hidden state of dynamic systems is crucial in many fields, from autonomous driving to smart city monitoring. Traditional Kalman Filters (KFs) excel at this but struggle with high-dimensional data (like images) and complex, unknown observation models. Existing methods either rely on accurate, often unavailable, analytical models or use purely data-driven approaches that can be sensitive to initialization, lack interpretability, and may not leverage available domain knowledge about system dynamics. This paper tackles the challenge of tracking from high-dimensional signals where the observation model is unknown, potentially nonlinear, and the system might be partially observable, all while aiming for real-time performance.

Methods

Latent-KalmanNet is built in a step-by-step manner, progressively addressing the challenges:

  1. Instantaneous Estimate (Step 1): A deep neural network (DNN) encoder is trained to map high-dimensional observations directly to an estimate of the observable state variables. This handles the unknown and complex observation model.
  2. Incorporating Evolution Model (Step 2): The state evolution function is used to provide a "prior" to the encoder. This helps the encoder leverage temporal correlations and improve its estimate, guiding the learning process.
  3. EKF in Latent Space (Step 3): The trained encoder's output is treated as a latent observation, and a classic Extended Kalman Filter (EKF) is used for tracking in this latent space. This step aims to incorporate temporal correlation and partial observability.
  4. Latent-KalmanNet (Step 4): The EKF is replaced with KalmanNet, a data-driven filtering approach that can handle non-Gaussian noise and unknown dynamics in the latent space. This module is trained jointly with the encoder, allowing them to mutually benefit. The encoder learns a latent representation that best facilitates tracking, while KalmanNet provides a prior for the encoder.

The training is performed using an alternating minimization strategy, where the encoder and KalmanNet modules are trained iteratively, leveraging the interpretability of the architecture.

Datasets

The paper uses two primary datasets for evaluation:

  • Pendulum Dataset: Generated from a simulated physical pendulum system. The observations are 28x28 grayscale images of the pendulum's movement, corrupted by additive Gaussian noise. This dataset is used to evaluate the contribution of each design step of Latent-KalmanNet.
  • Lorenz Attractor Dataset: Generated from a simulated chaotic Lorenz system. Observations are 28x28 grayscale images representing the system's state, corrupted by Salt and Pepper (S&P) noise. This dataset is used for comparative evaluation against other methods under various noise levels and model mismatches.

The paper does not introduce new benchmark datasets but uses generated data that mimics real-world challenges.

Key Insights and Ideas

  • Synergistic Learning: The core idea is that the encoder and the tracking module (KalmanNet) can learn to help each other. The encoder learns a latent space that is "easy" for the tracker, and the tracker provides useful priors for the encoder. This joint learning is key to overcoming the limitations of separate approaches.
  • Latent Representation for Tracking, Not Just Estimation: The paper shows that the latent representation learned by the encoder isn't necessarily the best estimate of the observable state itself. Instead, it's optimized to be a representation that facilitates accurate tracking, even if it's not a direct estimate of the ground truth.
  • Robustness to Model Mismatch: Latent-KalmanNet demonstrates significant robustness to mismatches in the state evolution model (e.g., using a simplified Taylor expansion) and sampling rates, outperforming methods that rely more heavily on accurate model knowledge.
  • Balancing Model-Based and Data-Driven: The approach effectively bridges the gap between traditional model-based filters (like KF/EKF) and purely data-driven deep learning methods. It leverages the interpretability and structure of model-based approaches while benefiting from the flexibility and learning power of deep networks.

Key Results

  • Improved Accuracy: Latent-KalmanNet consistently achieves lower Mean Squared Error (MSE) compared to benchmark methods (including pure encoders, encoder+prior, encoder+prior+EKF, and RKN) across various noise levels and system dynamics (Pendulum and Lorenz attractor). For instance, in the Lorenz attractor experiment with full domain knowledge, Latent-KalmanNet achieved an MSE of -7.94 dB, significantly better than RKN (-7.16 dB) and Encoder+Prior+EKF (-6.31 dB).
  • Faster Inference: Latent-KalmanNet offers faster inference times compared to both model-based EKF and the purely data-driven RKN. For the Lorenz attractor, Latent-KalmanNet achieved an inference time of 0.09 seconds, compared to 0.39 seconds for RKN and 0.36 seconds for Encoder+EKF.
  • Generalization: The method demonstrates good generalization to longer trajectories than those used during training.
  • Robustness: Latent-KalmanNet shows strong performance even with partial domain knowledge of the state evolution function and mismatches in sampling rates.

Limitations and Future Work

  • Computational Complexity: While faster than some benchmarks, the complexity still scales with the dimensions of the trainable modules.
  • Interpretability of Latent Space: While the overall framework is interpretable, the specific learned latent representation might not always have a clear physical meaning.
  • Future Work: The authors suggest exploring the properties of Latent-KalmanNet for extracting uncertainty estimates (as shown in [33]) and for unsupervised learning (as shown in [34]), which are valuable extensions for real-world applications. Further investigation into the learned latent space and its properties is also a potential avenue.

Summary updated

BibTeX
@article{buchnik2024latent,
  author = {Itay Buchnik and Guy Revach and Damiano Steger and Ruud J. G. van Sloun and Tirza Routtenberg and Nir Shlezinger},
  title = {{Latent-KalmanNet: Learned Kalman Filtering for Tracking From High-Dimensional Signals}},
  journal = {IEEE Trans. Signal Process.},
  year = {2024},
  doi = {10.1109/tsp.2023.3344360},
}
Figure