Moment Readouts: Reservoir Computers That Separate Drift, Diffusion, and Sensor Noise

Reservoir computing trains only a linear output layer on top of a fixed, randomly wired recurrent network. It is one of the cheapest ways to learn and forecast chaotic dynamics: one ridge regression replaces hours of backpropagation. But the standard recipe carries a hidden assumption. The readout is trained to predict the next observation, and then the network is run "closed loop," feeding its own predictions back in. That procedure treats the system as deterministic. Any mismatch between prediction and data is treated as error to be averaged away.

When the real system is kicked around by internal randomness, called process noise, this assumption fails in a specific way. The readout learns the conditional mean, so closed-loop rollouts collapse toward smooth, overly confident trajectories. They under-disperse the attractor, cannot report how uncertain the system is in different regions, and confuse random physical kicks with sensor error.

We propose to keep the reservoir fixed and add a small set of extra linear readouts, all fit in closed form on the same matrix of reservoir states. They regress the first and second moments of future increments at several time lags. Two scaling facts do the separating. The variance contributed by process noise grows linearly with the lag, while sensor noise contributes a lag-independent offset. From a single trajectory, with only part of the state observed and no retraining of the reservoir, this yields three things: a drift field, a state-dependent diffusion field, and a sensor-noise level. The recovered fields then drive stochastic closed-loop generation that should reproduce the system's long-run statistics, not just its mean path.

Core Claim

A fixed reservoir already contains what is needed to estimate a stochastic differential equation, meaning a model $dX_t = f(X_t)\,dt + g(X_t)\,dW_t$ with deterministic trend $f$, noise magnitude $g$, and Brownian motion $W_t$. What is missing is the right set of regression targets. Let $h_t$ be the reservoir state after reading observations $y_{\le t}$, sampled every $\Delta t$. Let $r_{t,k} = y_{t+k} - m_k(h_t)$ be the residual of a linear mean readout $m_k$ at lag $k$. For small $k\Delta t$ we expect

$$\mathbb{E}[\,r_{t,k} r_{t,k}^\top \mid h_t\,] \approx A(h_t) + k\,\Delta t\, g g^\top(\hat x_t) + O(k^2 \Delta t^2).$$

The intercept $A$ absorbs sensor noise plus the reservoir's own uncertainty about the hidden state. The slope is the diffusion, following from the linear-in-time variance of Brownian increments. The quadratic term captures drift curvature.

Each lag's second-moment readout is itself a ridge regression of residual outer products on $h_t$. Fitting the lag profile per state is then a tiny least-squares problem. The whole procedure reuses one state matrix, which is exactly the reuse property highlighted by Lukosevicius et al..

What is new here is neither stochastic reservoirs nor moment regression on its own. It is identifying drift, diffusion, and a sensor-noise bound from a single partially observed trajectory using only closed-form readouts on an untrained reservoir, and then using them for faithful stochastic generation.

Positioning

The prior work plays three roles.

Papers that motivate the gap. de Bezenac et al. and Hwang et al. show that deterministic predictors collapse turbulent or weather-like distributions into point predictions. We expect the same failure in deterministic closed-loop reservoirs, and we test for it directly.

Papers that constrain the design. Tripura et al. separate a Lagrangian drift from volatility using first and second moments. They report under 1% parameter error, but they take expectations over an ensemble of trajectories from shared initial conditions. We replace ensemble averages with conditional expectations given the reservoir state, which is what makes single-stream data usable. SDE Matching makes latent stochastic models fast to train, but it still trains deep score and velocity networks. We train nothing but linear maps.

Papers that supply the method, and the closest neighbors. - Random Controlled Differential Equations builds untrained continuous-time reservoirs that read rough paths and recover roughness properties such as the Hurst exponent. The path is treated as an external input, though, and is never decomposed into drift and diffusion. - ResCP uses reservoir-state similarity to reweight past residuals for conformal intervals. It calibrates uncertainty but cannot generate dynamics. Our second-moment readouts are a parametric counterpart that can drive simulation. - Euler State Networks supply one reservoir architecture we test. Their antisymmetric, non-dissipative design was built under a deterministic ODE assumption, and we examine whether it still helps once noise is injected.

On theory, Gonon and Ortega prove that linear readouts of fixed reservoirs can approximate filters of unbounded stochastic inputs. This licenses in principle the conditional-moment readouts we fit, though their result is non-constructive. Transport in reservoir computing shows that stability should be judged "on average" over input noise, a property it calls stochastic state contractivity. That is the right lens for the new risk we introduce: feeding injected noise back into the reservoir during generation.

Research Program

Estimator. We first build the estimator. We drive a fixed reservoir with the observed series and collect states $H$. We then fit mean readouts $m_k$ for $k = 1,\dots,K$ with $K$ around 5 to 10. Next we fit second-moment readouts on residual outer products, projected onto positive semidefinite matrices. Finally, for each state we fit the quadratic-in-$k$ profile above to read off $\hat g \hat g^\top$ and the intercept $A$.

The drift estimate is taken from the slope of the mean readouts across lags rather than from the lag-1 readout alone. This makes it less sensitive to sensor noise. We compare three reservoir types under an equal hyperparameter-search budget: a leaky echo state network, an antisymmetric Euler State Network, and an Euler–Maruyama variant.

Generation. Second, we build a stochastic closed-loop generator. At each step it advances the observation by the drift readout plus a Gaussian draw scaled by the diffusion readout, then feeds the result back into the reservoir.

Stability analysis. Third, we analyze when this loop stays stable. We estimate average contraction empirically: two copies of the reservoir, driven by the same noise sequence from different initial states, should converge. We relate this to spectral radius and noise magnitude, borrowing the average-contractivity criterion from Transport in reservoir computing.

Identifiability limits. Fourth, we characterize where the method breaks. Under partial observation the intercept mixes sensor noise with state uncertainty, so we report it only as an upper bound on sensor noise. We also map how the choice of $\Delta t$ trades discretization bias in the slope against estimator variance.

Baselines

Naive reference. The do-nothing reference is the standard deterministic reservoir: an echo state network with a ridge one-step readout, run closed loop, following the recipe going back to Jaeger. It uses the identical reservoir, states, and ridge penalty search. This is the baseline whose deterministic assumption we claim is wrong. Beating it means matching or improving short-horizon forecast error while substantially lowering the distance between generated and true long-run distributions.

Strongest learned competitor. This is a fully trained latent stochastic differential equation, in two forms: the adjoint-sensitivity version and SDE Matching. On 50-dimensional motion-capture gait data, that paper reports test mean squared error of 4.03 for the adjoint method and 4.50 for SDE Matching. We run both on the same trajectories with the same observed dimensions. We give them an equal hyperparameter-trial budget, and we report wall-clock time next to accuracy. Beating them means coming within their error range on MOCAP at a fraction of the training time. On synthetic systems, where the true drift and diffusion are known, it means recovering those fields more accurately.

Strongest classical competitor. This is conditional-moment regression of increments on an explicit polynomial library, in the spirit of stochastic SINDy. It is flagged as an open frontier in PySINDy and carried out with ensembles by Tripura et al.. We apply it three ways: to the full state, to delay coordinates of the observed channel, and with the same multi-lag correction we use. That last version isolates whether the reservoir itself contributes anything beyond the lag trick.

Ablations. Three ablations isolate the key ingredients:

  • Homoscedastic noise. Closed-loop generation with a single constant noise level fit from residuals, which isolates the value of state-dependent diffusion.
  • Single lag. A one-lag second-moment readout, which isolates the separation of sensor noise from diffusion.
  • No recurrence. A reservoir with its recurrent weights removed, leaving only random features of the current observation, which isolates the reservoir's memory.

Why not just…?

Why not just train a latent SDE end to end? It is the most expressive option and a real threat on accuracy. The MOCAP numbers above suggest trained stochastic models set a strong bar. The costs are also documented in SDE Matching: 30 hours for adjoint training on the pendulum video, gradients that become unstable over long horizons, and a learned diffusion that is not directly separated from sensor noise. Our experiments settle this by reporting accuracy against training time on the same data. If trained models win decisively on field recovery at matched observation, our claim narrows to cheap, streaming, low-data use.

Why not just run Kramers–Moyal regression on delay coordinates? Kramers–Moyal regression estimates drift from the average increment, and diffusion from the average squared increment, conditioned on the current state. This is the most serious threat. With the full state observed and sensor noise small, it should work well from a single trajectory, since a stationary chaotic trajectory visits its attractor densely. The reservoir should matter only when the state is partially observed and sensor noise is non-negligible. In that regime delay embeddings grow in dimension, polynomial libraries on them explode, and squared increments are biased upward by sensor noise. We test exactly this crossover. If delay-coordinate Kramers–Moyal with our multi-lag correction matches us everywhere, we will conclude that the lag trick, not the reservoir, is the contribution, and report it that way.

Why not just add residual-sized noise to a deterministic reservoir rollout? That is our homoscedastic ablation. It will fix gross under-dispersion. It cannot represent noise whose size depends on the state, as in a Duffing oscillator with multiplicative forcing or heteroskedastic exchange rates. It also still attributes sensor noise to the dynamics, which should over-disperse the generated attractor when sensors are noisy.

Why not just denoise first and fit a deterministic model? Smoothing removes exactly the quadratic variation, meaning the accumulated squared increments that encode diffusion. Rudy et al. note that sequential denoise-then-fit pipelines risk losing physical information, and their own model assumes a deterministic vector field. We include a smoothing-spline-plus-SINDy pipeline and expect it to report near-zero diffusion.

Proposed Experiments

Synthetic systems with ground truth. The core testbed is systems where drift and diffusion are known exactly. We use the stochastic Lorenz attractor used by SDE Matching; no reported accuracy number was found on Lacuna for it, only a 50x speedup. We also use the stochastically excited Duffing oscillator, pendulum, and 3-degree-of-freedom frame of Tripura et al.. There the reference point is under 1% parameter error obtained with ensembles, whereas we use a single trajectory.

Crossed factors. For each system we cross three factors:

  • Observation: full state versus one coordinate.
  • Sensor noise: 0%, 10%, and 25% of signal standard deviation, mirroring the noise levels in Rudy et al..
  • Trajectory length: a range of lengths.

Metrics. We report five measures:

  • relative error of drift and diffusion evaluated on a grid over the attractor;
  • relative error of the sensor-noise bound;
  • Wasserstein-1 distance between marginals of long generated runs and the true invariant distribution;
  • mismatch in autocorrelation;
  • coverage of multi-step predictive intervals.

Real data. On MOCAP we report test MSE against the 4.03 and 4.50 figures above, plus wall-clock time. On the Exchange series from ResCP we compare Winkler interval scores against ResCP itself. The file reports only relative gains for ResCP, so we rerun it rather than quote a number.

Decisive checks. Three results would decide the question:

  1. Scaling check. On single partially observed stochastic Lorenz and Duffing trajectories, the estimated slope should track the true diffusion as we vary the true noise level, and the intercept should track the injected sensor noise. If the slope instead absorbs sensor noise, the core identification fails.
  2. Generation check. Stochastic closed-loop generation should cut Wasserstein-1 error of the invariant distribution substantially relative to the deterministic reservoir, while keeping short-horizon forecast error equal. It should also beat the homoscedastic ablation on the multiplicatively forced systems.
  3. Crossover check. Against delay-coordinate Kramers–Moyal, we expect to lose or tie under full observation and to win under partial observation with sensor noise. Failing to find that region is the result that would falsify the proposal.