For decades, state estimation has been dominated by two distinct philosophies. On one hand, classical Bayesian filters—like the Kalman Filter (KF)—rely on elegant physical models to track hidden states through time. They are interpretable and mathematically optimal, but they brittle when faced with complex, non-linear dynamics or unknown noise statistics. On the other hand, Deep Learning offers unparalleled flexibility in pattern recognition but often behaves as a "black box," ignoring known physical laws and struggling with temporal consistency.
A growing body of research is merging these worlds. By embedding neural networks directly into the recursive "predict-update" cycle of Bayesian filters, researchers are creating hybrid architectures that leverage physical priors while learning to compensate for model inaccuracies from data.
The Hybrid Predict-Update Loop
At its core, a Bayesian filter operates in two steps: a prediction (using a motion model to guess the next state) and an update (using a measurement to correct that guess). In traditional settings, these steps require a perfectly defined transition matrix and precise noise covariance matrices ($Q$ and $R$).
In real-world scenarios, such as tracking human motion or predicting traffic flows, these matrices are rarely known. Researchers like Xi Xiong and Li Jin 0004 have shown that integrating deep learning into this loop allows the neural network to capture complex non-linear patterns while the classical filter acts as a "stabilizing effect," grounding the output in physical constraints Dynamic Prediction of Origin-Destination Flows Using Fusion Line Graph Convolutional Networks.
Learning the Kalman Gain
One of the most influential breakthroughs in this direction is KalmanNet, introduced by Guy Revach, Nir Shlezinger, and Ruud J. G. van Sloun. Instead of trying to analytically derive the "Kalman Gain"—the weight that determines how much to trust the measurement versus the prediction—KalmanNet uses a Recurrent Neural Network (RNN) to learn this gain directly from data KalmanNet: Neural Network Aided Kalman Filtering for Partially Known Dynamics.
This approach works because the RNN can recognize temporal dependencies and noise patterns that a static mathematical formula would miss. By maintaining the structure of the Kalman Filter, the model retains the inductive bias of state-space modeling, which makes it more data-efficient than a pure RNN. As Ting Gao demonstrates in traffic queue estimation, this allows the system to adapt to time-varying noise that is nearly impossible to model manually Q-Net: Transferable Queue Length Estimation via Kalman-based Neural Networks.
Neural Parameter Tuning and Hypernetworks
Beyond learning the gain, neural networks are being used to "tune" the filter's internal parameters in real-time. Kumar Pratik and Max Welling proposed the Hypernetwork Kalman Filter (HKF), where a neural network dynamically predicts transition matrices and noise statistics based on observed signal sequences Neural Augmentation of Kalman Filter with Hypernetwork for Channel Tracking.
This is particularly useful in high-mobility environments, such as 5G/6G beam tracking or drone racing. For instance, Giovanni Cioffi and Davide Scaramuzza developed a system where a deep learning module predicts relative displacement, which an Extended Kalman Filter then treats as a "virtual sensor" update to correct for the drift inherent in raw inertial data Learned Inertial Odometry for Autonomous Drone Racing.
Robustness Through Data Assimilation
In large-scale physical systems like weather forecasting or epidemiology, this fusion is often framed as Data Assimilation (DA). Rossella Arcucci and colleagues have demonstrated that using neural networks as "forward models" within an Ensemble Kalman Filter allows for continuous state adjustment based on noisy, real-time signals A novel approach for predicting epidemiological forecasting parameters based on real-time signals and Data Assimilation.
The intuition here is that the neural network provides the "global intuition" (identifying complex, non-linear trends), while the Bayesian filter provides the "local precision" (correcting for sensor noise and ensuring the state doesn't jump physically impossible distances). This synergy is now being formalized in software packages like TorchDA, which allow researchers to plug neural networks directly into variational and sequential filtering workflows TorchDA: A Python package for performing data assimilation with deep learning forward and transformation functions.
Why the Hybrid Approach Wins
The reason these hybrid models—often called Deep State Space Models (DSSM)—outperform standalone AI is their handling of uncertainty. Pure neural networks often struggle to quantify how "sure" they are of a prediction. By integrating with Bayesian frameworks:
- Heteroscedastic Noise Estimation: Networks can learn to predict the variance of the noise, allowing the filter to ignore measurements when the environment is chaotic Sampling-Resilient Multi-Object Tracking.
- Physical Consistency: Known geometric relationships (like Jacobian matrices in SLAM) can be preserved, while the neural network handles only the "unknown" statistical parts of the problem Split-KalmanNet: A Robust Model-Based Deep Learning Approach for SLAM.
- Unsupervised Adaptation: By minimizing the "innovation process" (the difference between what the filter predicted and what was actually observed), these models can learn to track hidden states without ever seeing ground-truth labels Unsupervised Learned Kalman Filtering.
As summarized by Rochelle Schneider and Massimo Bonavita, the field is moving away from "black-box" experimentation toward physics-integrated, operationally viable models that combine the best of both mathematical rigor and data-driven flexibility ESA-ECMWF Report on recent progress and research directions in machine learning for Earth System observation and prediction.