Resonator Reservoirs for Forecasting Chaos from Partial Observations

Reservoir computing trains only a linear readout on top of a fixed, randomly wired recurrent network. On chaotic systems it is fast and accurate, but only when the model sees the full state of the system. When it sees only a compressed view, such as the few dominant spatial modes of a turbulent field, it breaks down. In the Lorenz-96 and Kuramoto–Sivashinsky studies of Forecasting of Spatio-temporal Chaotic Dynamics with Recurrent Neural Networks, reservoirs diverged within one Lyapunov time on reduced-order data. A Lyapunov time is the natural time over which small errors in a chaotic system grow by a factor of e. Gated recurrent networks trained with backpropagation through time lasted nearly two Lyapunov times on the same data, though they needed hours of training rather than seconds.

We propose that this collapse is a memory problem rather than a capacity problem. When modes are discarded, the forecaster must infer them from the history of the modes it can see, which is the logic of a delay embedding. A dissipative random reservoir keeps a poorly organized trace of that history. A reservoir built from damped, driven oscillators is different: each unit is a resonator with its own frequency and decay time, and together they act like a bank of windowed sine transforms of the input (Neural Oscillators are Universal). The proposal is to build such resonator reservoirs, set their frequencies and damping from the measured spectrum of the observed signal instead of by blind search, and test three things. The first is whether they close the reduced-order gap without any backpropagation. The second is whether the gain survives a strong delay-embedding baseline. The third is whether how well the reservoir state predicts the discarded modes explains the result.

Core Claim

Our claim is that reservoir forecasts of partially observed chaos fail because the reservoir state holds too little linearly decodable information about the unobserved modes. A heterogeneous oscillator reservoir whose frequencies cover the observed power spectrum and whose decay times span up to a few Lyapunov times stores this information much better. The measurable consequence is that the reservoir's accuracy at decoding the discarded modes should predict how long its closed-loop forecasts stay valid, across architectures and observation levels.

We state up front what is not claimed. The linear part of a second-order oscillator layer is exactly a first-order system of twice the dimension, made of 2×2 rotation-and-decay blocks. Any benefit could therefore come from the choice of frequencies and decay times rather than from "second-order-ness" itself. The experiments are built to separate these two explanations. The one genuinely second-order structural difference is that in coRNN-style updates the nonlinearity drives acceleration, so its output passes through a resonator before reaching position. That difference is tested directly.

Relation to Prior Work

The mechanism comes from two papers. Random Oscillators Network for Time Series Processing introduced the untrained random oscillator reservoir (RON), a dense network of heterogeneous damped oscillators integrated with forward Euler. It reports Lorenz-96 normalized root-mean-squared error of 0.0016, against 0.0020 for a leaky echo state network and 0.24 for an LSTM. It evaluated full-state forecasting only and tuned stiffness and damping by search. Coupled Oscillatory Recurrent Neural Network (coRNN) supplies the update equation and an implicit-explicit integrator with bounded energy, but trains all weights. Like Neural Wave Machines, it reports that trained oscillator networks handle chaos poorly. We reuse both, with no recurrent training, and move to the reduced-order regime that neither paper touches.

The motivation and the bar to beat come from Forecasting of Spatio-temporal Chaotic Dynamics with Recurrent Neural Networks. It finds that first-order reservoirs collapse under reduced-order observation and that gated networks trained with backpropagation do not. Its result constrains us: if resonator reservoirs also fail within one Lyapunov time, the hypothesis is wrong. Data-driven prediction of a multi-scale Lorenz 96 chaotic system shows that reservoirs beat LSTMs at full observation, with a valid horizon of about 2.3 model time units against about 0.9. That confirms the weakness is specific to partial observation.

Two first-order reservoirs serve as controls rather than as sources of the method. Reservoir Computing by Discretizing ODEs (the Euler state network, or EuSN) uses an antisymmetric matrix so that dynamics neither decay nor blow up. It is the strongest first-order route to long memory, but its frequencies come from a random matrix rather than from the data. Random Controlled Differential Equations gives a continuous-time random-feature reservoir, but it is aimed at classifying rough paths rather than at closed-loop forecasting, so we do not use it as a main baseline.

Research Program

The model is a single-layer reservoir of $N$ oscillators updated with the implicit-explicit coRNN step. Recurrent, input and bias weights are random and fixed, and a ridge-regression readout maps the position and velocity states to the next observed modes. In autonomous forecasting the readout output is fed back as input.

The new ingredient is a rule for setting each oscillator's stiffness $\gamma_i$ and damping $\epsilon_i$ from training data. Natural frequencies $\sqrt{\gamma_i}$ are sampled from the empirical power spectrum of the observed modes. Damping is set so that the decay times $2/\epsilon_i$ are spread log-uniformly between one sampling step and a few Lyapunov times, where the Lyapunov time is estimated from the training trajectory. After this rule, the only free hyperparameters are a global input scale, a global recurrent scale and the ridge penalty. That is fewer than a leaky echo state network needs, which bears on the hyperparameter cost noted for Locally Connected Echo State Networks for Time Series Forecasting, up to 20 hours of CMA-ES search per dataset.

The diagnostic runs alongside the models. For every trained reservoir, we fit a separate ridge probe from the open-loop reservoir state to the modes that were discarded from the input, and record the held-out probe error. We then ask whether this "hidden-mode decodability" predicts closed-loop valid prediction time across all architectures and observation levels. This turns a qualitative story into a test that could fail.

Baselines

The naive reference point is persistence plus a linear autoregressive model on the observed modes. This shows how much any recurrent memory adds at each observation level.

The first strongest competitor is the BPTT-trained GRU and LSTM setup of Forecasting of Spatio-temporal Chaotic Dynamics with Recurrent Neural Networks. It reports a valid prediction time of nearly 2 Lyapunov times on reduced-order Lorenz-96, the only method in the file that survives that regime. We retrain it on identical trajectories and splits, with the same number of hyperparameter trials as the reservoirs. We report wall-clock time on the same GPU. Beating it means matching or exceeding its valid prediction time with paired confidence intervals over at least 100 test initial conditions, at a training cost one to two orders of magnitude lower. Matching it at that lower cost would already count as a result.

The second strongest competitor is a first-order leaky echo state network given an explicit delay embedding: the current observed modes plus $k$ lagged copies, with $k$ and the lag spacing tuned. This is the most direct alternative explanation, discussed below. It receives the same state dimension (counting $2N$ for the oscillator reservoir) and the same tuning budget.

Three further controls round out the comparison. A plain leaky echo state network reproduces the reported reduced-order collapse. RON with its original forward Euler step and searched parameters separates the effect of the spectrum rule and integrator from the effect of oscillators per se. EuSN is the strongest non-dissipative first-order reservoir.

The key ablation is a matched-spectrum first-order reservoir. It uses complex-diagonal (2×2 block) linear dynamics with exactly the same frequencies and decay times as the resonator reservoir, but the nonlinearity is applied to the state rather than to the acceleration. If this ablation equals the resonator reservoir, the contribution is the spectrum rule, not second-order structure, and we will report it that way. A second ablation draws frequencies and damping randomly over the same ranges, isolating the data-matched rule.

Why not just…?

Why not just feed a delay embedding to a standard echo state network? This is the real threat. Delay coordinates are the textbook remedy for partial observation, and a resonator bank may simply be an implicit delay embedding. The file contains no reduced-order result for delay-embedded reservoirs. We therefore run this baseline with a tuned lag structure and equal state size. If it matches the resonator reservoir, the paper's conclusion becomes "the reduced-order collapse reported by Vlachas et al. is a missing-history problem, fixable by delays." That is still a useful correction to the literature. The decodability probe would then tell us which design stores hidden-mode information more compactly per state dimension.

Why not just train a GRU? It already works, at nearly 2 Lyapunov times, per Forecasting of Spatio-temporal Chaotic Dynamics with Recurrent Neural Networks. However, it took hours where reservoirs took seconds, and the same comparison shows reservoirs reproduce long-term statistics well at full observation. The question is whether a few-minute, closed-form method can recover the gated networks' robustness. The matched-budget comparison answers this directly.

Why not just use an antisymmetric first-order reservoir or a bigger ESN? EuSN extends memory, but its frequencies come from a random antisymmetric matrix rather than the data. A bigger ESN adds width without organizing memory by timescale. Both are run at equal and at doubled state size. If neither approaches the resonator reservoir, while the matched-spectrum first-order ablation does, then the decisive factor is how memory timescales are allocated rather than size or non-dissipation.

Proposed Experiments

The main benchmarks follow Forecasting of Spatio-temporal Chaotic Dynamics with Recurrent Neural Networks. The first is 40-variable Lorenz-96 and the second is Kuramoto–Sivashinsky on the 512-point grid. Each is projected onto its leading singular-value modes, and the number of retained modes is swept from near-full down to a handful, tracing the full collapse curve rather than a single point. Full-state Lorenz-96 is a sanity check against RON's reported normalized error of 0.0016.

Metrics are valid prediction time in Lyapunov times and agreement of long-run statistics, namely the marginal distributions and the leading Lyapunov exponents of the autonomous rollouts. We also report training wall-clock and inference memory. For reduced-order Lorenz-96, the bar is the reported reservoir failure within 1 Lyapunov time and the gated-network value of nearly 2. For reduced-order Kuramoto–Sivashinsky, the file reports only that reservoirs "frequently diverged" and gated networks were more robust, with no numeric valid prediction time. The comparison there is against our own reruns.

The secondary benchmark is the three-tier multi-scale Lorenz-96 of Data-driven prediction of a multi-scale Lorenz 96 chaotic system, observing only the slow variables. This probes whether oscillators with long decay times recover the influence of the unobserved fast tiers.

Four checks decide the outcome. The hypothesis is supported only if the resonator reservoir passes the 1-Lyapunov-time collapse point by a clear margin at low mode counts and beats the delay-embedded ESN at equal state size. A third check is that hidden-mode decodability correlates with valid prediction time across all models and observation levels; a weak correlation would falsify the memory explanation even if the forecasts improve. Finally, the matched-spectrum and random-spectrum ablations determine whether credit goes to second-order structure, to the data-matched timescales, or to neither. We also count how many hyperparameter trials each method needs to reach 90 percent of its best valid prediction time, to test whether the spectrum rule really reduces search cost.