Abstract
Machine learning techniques offer an effective approach to modeling dynamical systems solely from observed data. However, without explicit structural priors—built-in assumptions about the underlying dynamics—these techniques typically struggle to generalize to aspects of the dynamics that are poorly represented in the training data. Here, we demonstrate that reservoir computing—a simple, efficient, and versatile machine learning framework often used for data-driven modeling of dynamical systems—can generalize to unexplored regions of state space without explicit structural priors. First, we describe a multiple-trajectory training scheme for reservoir computers that supports training across a collection of disjoint time series, enabling the effective use of available training data. Then, applying this training scheme to multistable dynamical systems, we show that reservoir computers trained on trajectories from a single basin of attraction can achieve out-of-domain generalization by capturing system behavior in entirely unobserved basins.
Introduction
Modeling complex dynamical systems, such as weather patterns or neural activity, typically involves a trade-off between knowledge-based models (grounded in physical equations) and black-box models (learned from data). While black-box models like neural networks offer flexibility, they often fail at "out-of-domain generalization." For instance, a model trained on a pendulum's small oscillations may fail to predict its behavior when swinging over the top. This limitation is particularly acute in multistable systems, which can settle into different long-term attractors depending on initial conditions. Conventional wisdom suggests that a data-driven model cannot predict an unobserved attractor if it has only seen data from a different one. This research challenges that assumption by demonstrating that Reservoir Computing (RC) can infer global system dynamics from localized observations.
Methodology
The study employs Reservoir Computing, a recurrent neural network architecture where internal connections remain fixed and random. Only the output layer weights are trained, typically via linear regression, making it computationally efficient.
A critical innovation presented is a multiple-trajectory training scheme. Rather than training on a single continuous time series, the RC is trained on a set of disjoint, short trajectories. This allows the model to capture "transient" dynamics—the paths a system takes as it approaches an attractor. These transients contain vital information about the underlying vector field, effectively providing the "rules of motion" for the entire state space.
The framework was tested on four benchmark systems: the Duffing oscillator, a multi-well system, a magnetic pendulum with fractal basins, and a multistable Lorenz-like chaotic system.
Results and Insights
The core finding is that transient trajectories are "information goldmines." Even if a model never observes a specific attractor, the way the system moves near the boundaries of its known environment provides sufficient mathematical clues to reconstruct the global map.
In the Duffing system, an RC trained on just 10 trajectories from a single basin correctly identified the location and nature of a second, entirely unobserved attractor. For the magnetic pendulum, the model achieved 66% accuracy in predicting basin settlement across the entire state space, significantly outperforming baseline models. Most notably, in the Lorenz-like system, the RC reconstructed the geometry of a second chaotic lobe after being trained only on the first, indicating it had captured the system's fundamental symmetries.
The researchers also noted that standardizing input data—a common machine learning practice—can sometimes hinder generalization in systems with separated basins. Using "raw" inputs allowed the RC to better recognize distant attractors.
Discussion and Future Work
The success of RC in this context is attributed to an implicit inductive bias. The simplicity of RC, combined with regularized linear regression, tends to find the most parsimonious mathematical representation of the data, which often aligns with the true underlying physics.
However, limitations remain. Generalization depends heavily on data diversity; if training trajectories do not sufficiently sample the transients, the model cannot "see" other basins. Furthermore, while the results are empirically strong, a rigorous mathematical theory explaining how RC extrapolates these vector fields is still being developed. Future research will explore whether these capabilities scale to high-dimensional, real-world systems like global power grids or climate models.
Summary updated