Abstract
State estimation refers to determining the states of a dynamical system that starts from a noisy initial condition and evolves under process noise, based on noisy measurements and a known system model. For linear dynamical systems with white Gaussian noises of known mean and variance, Kalman filtering is a well-known method that leads to stable error dynamics for detectable systems. There are some non-optimal extensions to nonlinear systems. Recent work has used neural networks to develop estimators for nonlinear systems that optimize a criterion. Stability of the error dynamics is even more important than optimality. Jordan recurrent neural networks (JRNs) have a structure that mimics that of a dynamical system and are thus appealing for estimator design. We show that a JRN performs better than an extended Kalman filter(EKF) and unscented Kalman filter(UKF) for several examples. The main contribution of this paper is an input-to-state stability analysis of the error dynamics of JRNs. The stability of the error dynamics of several examples is shown.
Introduction
State estimation—the process of calculating the internal state of a system based on noisy sensor measurements—is a cornerstone of control engineering. For linear systems, the Kalman Filter (KF) is the gold standard. However, most real-world systems are nonlinear. Traditional solutions like the Extended Kalman Filter (EKF), which linearizes the system, or the Unscented Kalman Filter (UKF), which uses statistical sampling, often struggle with highly nonlinear dynamics or carry heavy computational costs.
While researchers have turned to Recurrent Neural Networks (RNNs) to solve this, a major hurdle remains: Stability. In control theory, it isn't enough for an estimator to be accurate; one must prove that the estimation error won't grow boundlessly over time. Proving this for standard RNNs (like Elman networks) is notoriously difficult due to their complex hidden-state dependencies.
Methods: Jordan Recurrent Neural Networks
The authors propose using Jordan Recurrent Neural Networks (JRNs) specifically for state estimation.
Architecture
Unlike Elman RNNs, which feed hidden states back into themselves, JRNs feed the output of the previous step back into the hidden layer. This structure closely mimics the mathematical form of physical dynamical systems, making them more "natural" estimators for state observation. This architecture is visualized below, showing both the compact and unfolded representations.
Stability Analysis
The core contribution is an Input-to-State Stability (ISS) analysis. The authors define the "error system" (the difference between the true state and the JRN's estimate) and seek a Lyapunov function—a mathematical "energy" function that proves the error will decay or stay bounded. To find these Lyapunov functions for nonlinear systems, they use a "counterexample-guided" approach. They train a small auxiliary neural network to represent the Lyapunov function and use SMT (Satisfiability Modulo Theories) solvers (specifically the dReal tool) to formally verify that the stability conditions hold across the entire state space.
Experimental Evaluation
The researchers evaluated their approach using synthetic datasets generated by discretizing three classic Ordinary Differential Equations (ODEs):
- Mass-Spring Damper: A standard linear system used to compare JRNs against the optimal Kalman Filter.
- Down Pendulum: A nonlinear system used to test accuracy against EKF and UKF.
- Reversed Van der Pol Oscillator: A complex nonlinear system characterized by limit cycles.
Data was generated with Gaussian process and measurement noise, split into 80% training, 10% validation, and 10% testing sequences.
Key Results
The JRN-based estimator consistently met or exceeded the performance of classical filters:
- Linear Performance: In the mass-spring example, the JRN achieved an RMSE of 0.5268, nearly matching the optimal Kalman Filter's 0.5132.
- Nonlinear Superiority: For the pendulum system, the JRN (RMSE 0.1298) significantly outperformed both the EKF (RMSE 0.1594) and the UKF (RMSE 0.1693). This performance gap is illustrated in the error dynamics plot below.
- Efficiency: While training a JRN takes time (e.g., 101 seconds for the pendulum), its inference time is much lower than the iterative sampling required by a UKF.
- Proven Stability: The authors successfully synthesized and verified ISS-Lyapunov functions for all three systems, providing mathematical proof that the estimation errors remain bounded.
Conclusion
By using SMT solvers to verify a neural network's stability, this work provides a rigorous framework to move neural estimators into safety-critical engineering applications. While the current study focuses on low-dimensional, bias-free networks, it demonstrates that JRNs can offer a superior balance of accuracy, efficiency, and theoretical stability compared to traditional nonlinear filters. Future work will explore scaling this analysis to higher-dimensional systems and more complex architectures like LSTMs.
Summary updated