Modeling Latent Non-Linear Dynamical System over Time Series

Ren Fujiwara, Yasuko Matsubara, Yasushi Sakurai

Abstract

Discovering the underlying mathematical equations of complex dynamical systems from noisy time-series data is a fundamental challenge in science and engineering. Traditional methods like Sparse Identification of Non-linear Dynamics (SINDy) often falter when faced with high noise levels or temporal dependencies. This paper introduces LaNoLem (Latent Non-Linear equation modeling), a framework that shifts system identification into a latent space. By simultaneously estimating hidden states and governing equations, LaNoLem effectively decouples signal from noise, achieving state-of-the-art performance in both equation discovery and long-term forecasting.

Introduction

System identification, or symbolic regression, aims to uncover the physical laws governing a system directly from observational data. While tools like SINDy have popularized this approach by seeking sparse representations of dynamics, they possess two critical vulnerabilities. First, they are highly sensitive to noise because they typically rely on numerical differentiation of raw data. Second, they often treat observations as independent snapshots, ignoring the inherent temporal continuity of dynamical systems.

LaNoLem addresses these issues by modeling the system in a latent space, ensuring that the estimated states are temporally consistent and the discovered equations are robust to observational noise.

Methodology: The LaNoLem Framework

The core of LaNoLem is a generative model that assumes observations are manifestations of underlying latent states $s$ evolving according to a combination of linear and non-linear dynamics.

Overview of the LaNoLem framework, illustrating the decomposition of original data into latent states and the modeling of their dynamics through linear (A) and non-linear (F) components.

Latent State Modeling

The model decomposes the dynamics into three parts:

  1. Linear Component ($A$): Captures stable, long-term trends.
  2. Non-Linear Component ($F$): Utilizes a polynomial library to capture complex, potentially chaotic behaviors.
  3. Offset ($b$): Accounts for constant shifts in the dynamics.

Optimization via Alternating Minimization

To solve for both the hidden states and the model parameters, LaNoLem employs an iterative two-step process:

  • Inference Step: An Extended Kalman Filter (EKF) is used to estimate the most likely latent trajectory given the current model. This step enforces temporal consistency, effectively filtering out point-wise noise.
  • Learning Step: Model parameters are updated using Elastic-Net regularization. This promotes "sparsity," ensuring the discovered equations are interpretable and use only the most relevant terms from the polynomial library.

The model's complexity is automatically tuned using the Minimum Description Length (MDL) principle, balancing fit accuracy with model simplicity.

Experimental Evaluation

The researchers benchmarked LaNoLem against several state-of-the-art methods, including STLSQ, SSR, and MIOSR, using two primary data sources:

  • dysts Database: A comprehensive set of 71 chaotic systems, providing a rigorous test for predicting systems where small errors amplify exponentially.
  • Google Trends: Real-world search data for seasonal keywords to test the model's ability to extract meaningful latent components from human-generated data.

Key Results and Discussion

LaNoLem demonstrated significant advantages in robustness and accuracy, particularly as data quality degraded.

Comparative performance of LaNoLem against baselines across 71 datasets. The plots show coefficient and prediction errors at varying noise levels (5%, 25%, 50%), with critical difference diagrams confirming LaNoLem's statistical superiority.

Robustness to Noise

While all methods performed well at low noise levels (5%), LaNoLem's performance remained stable even as noise increased to 50%. In contrast, baseline methods saw their accuracy plummet, as they struggled to differentiate between system dynamics and observational noise.

Prediction and Imputation

LaNoLem consistently achieved the lowest prediction errors across the 71-dataset benchmark. Notably, in the highest noise scenario (50%), it outperformed all competitors on every single dataset. Furthermore, by modeling the latent trajectory, the framework can accurately impute missing data points in chaotic systems where standard interpolation fails.

Conclusion

LaNoLem represents a significant step forward in automated physics discovery. By leveraging latent space modeling and temporal consistency, it overcomes the noise sensitivity that plagues traditional symbolic regression. Future work will explore expanding the library of basis functions beyond polynomials and optimizing the algorithm for high-dimensional real-time applications.

Summary updated

Abstract

We study the problem of modeling a non-linear dynamical system when given a time series by deriving equations directly from the data. Despite the fact that time series data are given as input, models for dynamics and estimation algorithms that incorporate long-term temporal dependencies are largely absent from existing studies. In this paper, we introduce a latent state to allow time-dependent modeling and formulate this problem as a dynamics estimation problem in latent states. We face multiple technical challenges, including (1) modeling latent non-linear dynamics and (2) solving circular dependencies caused by the presence of latent states. To tackle these challenging problems, we propose a new method, Latent Non-Linear equation modeling (LaNoLem), that can model a latent non-linear dynamical system and a novel alternating minimization algorithm for effectively estimating latent states and model parameters. In addition, we introduce criteria to control model complexity without human intervention. Compared with the state-of-the-art model, LaNoLem achieves competitive performance for estimating dynamics while outperforming other methods in prediction.
BibTeX
@inproceedings{fujiwara2025modeling,
  author = {Ren Fujiwara and Yasuko Matsubara and Yasushi Sakurai},
  title = {{Modeling Latent Non-Linear Dynamical System over Time Series}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i11.33269},
}
Figure