Abstract
Discovering the underlying mathematical equations of complex dynamical systems from noisy time-series data is a fundamental challenge in science and engineering. Traditional methods like Sparse Identification of Non-linear Dynamics (SINDy) often falter when faced with high noise levels or temporal dependencies. This paper introduces LaNoLem (Latent Non-Linear equation modeling), a framework that shifts system identification into a latent space. By simultaneously estimating hidden states and governing equations, LaNoLem effectively decouples signal from noise, achieving state-of-the-art performance in both equation discovery and long-term forecasting.
Introduction
System identification, or symbolic regression, aims to uncover the physical laws governing a system directly from observational data. While tools like SINDy have popularized this approach by seeking sparse representations of dynamics, they possess two critical vulnerabilities. First, they are highly sensitive to noise because they typically rely on numerical differentiation of raw data. Second, they often treat observations as independent snapshots, ignoring the inherent temporal continuity of dynamical systems.
LaNoLem addresses these issues by modeling the system in a latent space, ensuring that the estimated states are temporally consistent and the discovered equations are robust to observational noise.
Methodology: The LaNoLem Framework
The core of LaNoLem is a generative model that assumes observations are manifestations of underlying latent states $s$ evolving according to a combination of linear and non-linear dynamics.
Latent State Modeling
The model decomposes the dynamics into three parts:
- Linear Component ($A$): Captures stable, long-term trends.
- Non-Linear Component ($F$): Utilizes a polynomial library to capture complex, potentially chaotic behaviors.
- Offset ($b$): Accounts for constant shifts in the dynamics.
Optimization via Alternating Minimization
To solve for both the hidden states and the model parameters, LaNoLem employs an iterative two-step process:
- Inference Step: An Extended Kalman Filter (EKF) is used to estimate the most likely latent trajectory given the current model. This step enforces temporal consistency, effectively filtering out point-wise noise.
- Learning Step: Model parameters are updated using Elastic-Net regularization. This promotes "sparsity," ensuring the discovered equations are interpretable and use only the most relevant terms from the polynomial library.
The model's complexity is automatically tuned using the Minimum Description Length (MDL) principle, balancing fit accuracy with model simplicity.
Experimental Evaluation
The researchers benchmarked LaNoLem against several state-of-the-art methods, including STLSQ, SSR, and MIOSR, using two primary data sources:
dystsDatabase: A comprehensive set of 71 chaotic systems, providing a rigorous test for predicting systems where small errors amplify exponentially.- Google Trends: Real-world search data for seasonal keywords to test the model's ability to extract meaningful latent components from human-generated data.
Key Results and Discussion
LaNoLem demonstrated significant advantages in robustness and accuracy, particularly as data quality degraded.
Robustness to Noise
While all methods performed well at low noise levels (5%), LaNoLem's performance remained stable even as noise increased to 50%. In contrast, baseline methods saw their accuracy plummet, as they struggled to differentiate between system dynamics and observational noise.
Prediction and Imputation
LaNoLem consistently achieved the lowest prediction errors across the 71-dataset benchmark. Notably, in the highest noise scenario (50%), it outperformed all competitors on every single dataset. Furthermore, by modeling the latent trajectory, the framework can accurately impute missing data points in chaotic systems where standard interpolation fails.
Conclusion
LaNoLem represents a significant step forward in automated physics discovery. By leveraging latent space modeling and temporal consistency, it overcomes the noise sensitivity that plagues traditional symbolic regression. Future work will explore expanding the library of basis functions beyond polynomials and optimizing the algorithm for high-dimensional real-time applications.
Summary updated