Wind and solar output is hard to plan around for a specific reason. Most of the time it is fairly predictable, and then, at a cloud edge or a gust front, forecasts fail within minutes. Grid operators hedge with a fixed error distribution, sized either for the average day or for the worst one. Chaos theory measures exactly the property operators need: how fast nearby states of a system drift apart. That rate, the Lyapunov exponent, sets how quickly forecast error grows. Renewable forecasting papers do compute it, but only as a single number for an entire dataset, and they then discard it or feed it into a network as a constant column. This proposal measures the stretching rate locally, at the current state, from the derivative of a learned dynamics model. It then uses that local rate to set how wide forecast intervals should be at each lead time and how much reserve a dispatcher should hold right now.
In two sentences: we rebuild a hidden state from the lagged history of one power or irradiance sensor and learn a smooth model of how that state evolves. We then read off how strongly the model amplifies small errors around the current state, and widen or shrink the forecast interval accordingly. The test is whether this physically motivated signal predicts upcoming forecast failures, especially ramp events, better than the cheap signals operators already have, such as recent volatility or a neural quantile forecaster. The question also matters if the answer is no, because a negative result would tell the field that Lyapunov features in renewable forecasting are decorative.
Core Claim
Phase space reconstruction is the classical trick of stacking lagged copies of one measured signal, $[s(t), s(t-\tau), \dots, s(t-(d-1)\tau)]$, so that the stack traces out a shape equivalent to the system's full hidden state. On a series reconstructed this way, the finite-time Lyapunov exponent at the current state carries information about forecast error over the next $h$ steps. Recent volatility, recent ramp magnitude and learned quantile models do not capture that information. Here "finite-time" means the stretching rate over a short window starting from where the system is now, rather than averaged over all time. The signal should matter most at the onset of ramps, where backward-looking volatility is still low but the dynamics are already diverging. We compute the exponent from the Jacobian of a learned flow $\Phi_T$, the map that carries a reconstructed state forward by time $T$:
$$\lambda_T(\mathbf{x}) = \frac{1}{T}\log \big\| D\Phi_T(\mathbf{x}) \big\|_2 ,$$
where $\mathbf{x}$ is the current delay vector and $D\Phi_T$ is the matrix of partial derivatives of the flow. The interval half-width at lead $h$ is then
$$w_h(\mathbf{x}) = c_\alpha\, \sigma_0(\mathbf{x})\, e^{\lambda_T(\mathbf{x})\, h}.$$
Here $\sigma_0$ is the one-step error scale, $c_\alpha$ is a coverage constant fitted by split-conformal calibration on held-out data, and the exponential form follows from the definition of divergence, $\delta(t)\approx\delta_0 e^{\lambda t}$. The novelty does not lie in using delay embeddings for wind power or in confirming chaos. Both have been done. It lies in turning a state-dependent divergence rate into calibrated, horizon-dependent uncertainty and then into reserve decisions.
Relation to Prior Work
Two wind papers motivate the gap. Combined Prediction of Wind Power with Chaotic Time Series Analysis fixes the delay at $\tau=6$ and the dimension at $m=8$ using classical heuristics. It reports a largest Lyapunov exponent of 0.2548 as proof of chaos and then ensembles neural networks across $m\in\{7,8,9\}$. Research on Energy Storage Configuration Optimization goes further. It computes one global exponent with a simplified Wolf method, concatenates it with sliding mean and standard deviation into a bidirectional GRU, and passes a point forecast to a mixed-integer transformer-switching model. In both papers the exponent is a single scalar that never varies with the state. In the second, the dispatch stage receives no uncertainty at all. We keep their setting and replace the scalar with a state-dependent quantity that reaches the optimizer.
The method comes from three sources. Model adaptive phase space reconstruction (MAPSR) learns real-valued delays jointly with a Neural ODE, a neural network that defines a continuous-time vector field. On Lorenz it extends valid prediction to 7–8 Lyapunov times, against 2–3 for heuristic embeddings. It has only been tested on synthetic systems and a combustor, and it never uses its learned vector field to estimate local divergence. We reuse its differentiable embedding and add the Jacobian readout. Grigoryeva et al. show that iterated forecast error in reservoir computers grows at the trained model's top Lyapunov exponent. That result is the theoretical warrant for reading error growth from the model's own Jacobian, and we apply it locally rather than globally. LETS Forecast (DeepEDM) and Attraos supply strong embedding-based point forecasters. DeepEDM has been evaluated on Solar-Energy. Neither outputs a divergence rate, and both fix the embedding parameters by heuristic.
Two further results constrain the design. Chaos as an interpretable benchmark finds that forecast error scales with the Lyapunov exponent across 131 systems, which is evidence the signal is real at the dataset level. The same benchmark warns that pointwise metrics dominate evaluation, so we score calibration and tails, not only RMSE. On the operations side, Optimal Dispatch of Residential Photovoltaic Inverters shows that risk-neutral dispatch curtails 27.83 kWh where a 95% Conditional Value-at-Risk (CVaR) formulation curtails 44.05 kWh. CVaR is the expected loss in the worst tail of outcomes. That paper uses a static error distribution, and we make the distribution state-dependent. The Value of Multi-stage Stochastic Programming shows that rolling-horizon policies capture nearly all multi-stage value, so we use a rolling-horizon CVaR dispatch rather than a full multi-stage program.
Research Program
The pipeline has three stages. In the first stage we remove the deterministic diurnal cycle. For solar we divide irradiance or power by a clear-sky model to get the clear-sky index, following the logic of Modified Auto Regressive Technique. For wind we use power directly, restricted to the operating range. We deliberately avoid band-pass decompositions such as VMD and CEEMDAN, which Xing et al. and Wang et al. rely on. Those methods strip out the high-frequency content that a delay embedding needs, which is the tension the file identifies. A secondary experiment tests whether decomposing first corrupts the divergence estimate. Dividing by a clear-sky model removes the daily cycle without filtering out fast fluctuations.
In the second stage we fit a MAPSR-style embedding and Neural ODE to the normalized series, with a penalty on Jacobian magnitude so the vector field is not falsely rough on noisy data. We then compute $\lambda_T(\mathbf{x})$ at every forecast origin, using automatic differentiation through the ODE solver over windows $T$ matched to the lead times of interest. As a model-free cross-check we also compute a neighbor-based local exponent. This version finds historical analogs of the current delay vector and measures how their futures spread apart. It isolates whether the learned Jacobian adds anything beyond analog spread.
In the third stage the interval widths $w_h(\mathbf{x})$, calibrated by split conformal prediction, define per-interval error scenarios. These scenarios enter a rolling-horizon CVaR reserve and curtailment problem. The problem is modeled on the Dall'Anese inverter dispatch formulation on a reimplemented 20-house feeder driven by solar profiles, and on a simple reserve-sizing problem for a wind plant.
Baselines
The naive reference is smart persistence, which repeats the current clear-sky index, paired with a static empirical error quantile at each lead time. This is the "operator does nothing new" point. On SIRTA, SIAT reports a persistence RMSE of 93.33 W/m² against its own 76.94 W/m² at one step. That gives the scale of the gap any time-series method faces against image-based models.
The strongest uncertainty competitor is a learned quantile forecaster with a rolling-volatility input. Concretely, this is the LSTM quantile regression model from Ultra-short-term solar power forecasting, which reports point R² 0.9786 on HKUST rooftop data. To it we add rolling mean, rolling standard deviation and recent ramp magnitude as inputs, the same statistics that Energy Storage Configuration Optimization feeds its BiGRU, and we conformalize it to the same nominal coverage. We found no reported result on Lacuna for interval coverage, pinball loss or continuous ranked probability score (CRPS) on SIRTA, NREL SRRL, HKUST or the WIND Toolkit. This baseline will therefore be trained by us, with the same lookback window, the same splits and the same hyperparameter search budget, measured in GPU-hours and trials, as our model.
The strongest point forecasters for context are DeepEDM and Attraos, run with their published embedding settings. We wrap each in the same conformal calibration, scaled by its own recent residuals, so we can see whether a better point model alone closes the gap.
Two ablations isolate the key ingredient. In the first, we replace $\lambda_T(\mathbf{x})$ with the single global exponent, which is the prior-work setting. In the second, we replace it with the neighbor-spread estimate. Beating the strongest baseline means three things. Pinball loss and CRPS at 10–60 minute leads must fall with bootstrap confidence intervals excluding zero. Conditional coverage on ramp-onset intervals must be closer to nominal. In dispatch, constraint violations must be equal or fewer at lower curtailment or reserve cost. For dispatch, the reference figures are the 27.83 vs 44.05 kWh from Dall'Anese et al., which we will reproduce before comparing.
Why not just…?
Why not just feed rolling volatility into the forecaster? This is the most serious threat. Volatility already predicts much of the error variance, and the energy-storage paper already uses rolling standard deviation. Volatility looks backward, though. At the start of a ramp, the recent window can be calm while the reconstructed state sits in a region where nearby trajectories separate quickly. The decisive test is a partial-correlation and nested-model analysis. If adding $\lambda_T$ to a model that already has volatility and ramp features does not reduce held-out pinball loss on ramp-onset intervals, we will report that the local exponent is redundant in this setting.
Why not just let a quantile or mixture-density network learn heteroscedastic error directly? Such networks can in principle learn any state-dependent width. In practice they are trained at a fixed set of lead times. Rare ramps are thin in the training data, and SIAT notes exactly this rarity problem for extreme irradiance. The exponential-growth form gives a structured extrapolation across lead times from one local rate. We will test this by holding out the longest lead from training for both methods and comparing tail calibration there.
Why not just use sky cameras, satellites, or weather foundation models? For point accuracy they win. ECLIPSE reaches 24.0% forecast skill at 10 minutes on SIRTA, and Baguan-solar cuts RMSE by 16.08% over satellite-only models. We do not claim to beat them on RMSE. Our method needs only the inverter or SCADA log, so it applies at the many sites without cameras, and it addresses calibrated uncertainty, which those models do not output. If time allows, we will apply the divergence-based widths around an image model's point forecast to test whether the two combine.
Why not just use a static CVaR distribution? Static CVaR already fixes risk-neutral under-procurement. What it cannot do is reallocate the reserve. It carries the same buffer on calm and turbulent intervals. The dispatch experiment measures whether state-dependent scenarios reach the same violation rate with less total curtailment. If they do not, the operational case fails, regardless of how well the forecasts are calibrated.
Proposed Experiments
The first check is on synthetic data, where the true answer is known. On Lorenz and a subset of the 131-system benchmark with added noise, we test whether $\lambda_T$ from the learned Neural ODE tracks the true finite-time exponent computed from the governing equations. If it does not track under realistic noise, the real-data results cannot be interpreted. The second check is the redundancy test on real data. We use pyranometer series from SIRTA and NREL SRRL, PV power from HKUST, and turbine power from the NREL WIND Toolkit. For each, we measure the incremental predictive value of $\lambda_T$ for future absolute error after controlling for volatility features, stratified by ramp events defined as clear-sky-index or power changes above a threshold. The third check scores probabilistic forecasts against all baselines and ablations, using pinball loss, CRPS, and marginal and ramp-conditional coverage. The fourth check is the rolling-horizon CVaR dispatch on the reimplemented feeder and the wind reserve problem, reporting curtailment, reserve cost and violation counts.
A secondary experiment runs the pipeline with and without a prior VMD decomposition. It tests whether decomposition distorts the estimated divergence, which bears directly on the dispute between the signal-decomposition and phase-space approaches. The project succeeds if the second, third and fourth checks all favor the local exponent. It produces a clean, publishable negative result if the second check shows no incremental value.
References
- Combined Prediction of Wind Power with Chaotic Time Series Analysis
- Research on Energy Storage Configuration Optimization Method for Wind Farm Substations Based on Wind Power Fluctuation Prediction Integrating Chaotic Features and Bidirectional Gated Recurrent Units
- Model adaptive phase space reconstruction
- Forecasting causal dynamics with universal reservoirs
- LETS Forecast: Learning Embedology for Time Series Forecasting
- Attractor Memory for Long-Term Time Series Forecasting: A Chaos Perspective
- Chaos as an interpretable benchmark for forecasting and data-driven modelling
- Optimal Dispatch of Residential Photovoltaic Inverters Under Forecasting Uncertainties
- The Value of Multi-Stage Stochastic Programming in Risk-Averse Unit Commitment Under Uncertainty
- Modified Auto Regressive Technique for Univariate Time Series Prediction of Solar Irradiance
- A New Combined Prediction Model for Ultra-Short-Term Wind Power Based on Variational Mode Decomposition and Gradient Boosting Regression Tree
- Ultra-short-term solar power forecasting by deep learning and data reconstruction
- Solar Irradiance Anticipative Transformer
- ECLIPSE : Envisioning Cloud Induced Perturbations in Solar Energy
- Integrating Weather Foundation Model and Satellite to Enable Fine-Grained Solar Irradiance Forecasting