A battery scheduler that plans against a load forecast does not care about any single hour's error. It cares about the running sum of errors, because every unforeseen kilowatt-hour is drawn from, or pushed into, its store of charge. If forecast errors are independent from hour to hour, that running sum spreads like a random walk, growing as the square root of the window length. If the errors instead have long memory, so that an under-forecast hour tends to be followed by more under-forecast hours days later, the sum spreads faster, as $n^{H}$ with a Hurst exponent $H > 0.5$. Every safety margin built on the square-root assumption is then too small at exactly the horizons batteries operate on. This note proposes to measure that exponent on the errors of today's best load forecasters, to explain where it comes from dynamically, and to size battery and reserve margins from it.
The dynamical explanation is the substantive new piece. Recent Koopman-operator models of grid load break the series into a small set of oscillating modes for the daily, weekly and seasonal cycles and treat what remains as structureless "innovation noise" (Characterizing the load profile in power grids by Koopman mode decomposition of interconnected dynamics). A Koopman operator is the linear operator that advances any measured quantity of a nonlinear system one step in time. We argue that the leftover noise is the continuous part of that operator's spectrum: the smeared, non-peaked distribution of frequencies that mixing and chaotic dynamics produce. The shape of that continuous spectrum near zero frequency fixes the Hurst exponent that detrended fluctuation analysis would measure. If this holds, one analysis both separates calendar cycles cleanly from long-memory fluctuation and says, conditional on the current state of the grid, how quickly accumulated errors will spread.
Core Claim
Our claim has three parts, and each can fail on its own. The first is empirical. The out-of-sample errors of strong load forecasters, run on real electricity data, have a Hurst exponent measurably above 0.5 over windows of several hours to several days, once calendar periodicities are removed properly. The second is methodological. Removing those periodicities with an estimator that separates the discrete from the continuous parts of the Koopman spectrum, instead of fitting polynomials or Fourier filters, gives less biased estimates of $H$. It also allows $H$ to be estimated conditionally on the region of the reconstructed state space the grid currently occupies. The third is operational. Battery and reserve margins that scale as $n^{H(\text{state})}$ lower the rate of constraint violations at equal or lower operating cost than margins scaled by $\sqrt{n}$, by a single global $H$, or by uncorrected conformal intervals.
Two relations tie these parts together. Let $e_t$ be the forecast error at hour $t$ and let $S_n = \sum_{t=1}^{n} e_t$ be its running sum over a window of $n$ hours. $S_n$ is exactly the integrated profile that detrended fluctuation analysis builds, and it is also, up to efficiency factors, the state-of-charge deviation an uncorrected battery schedule accumulates. Long memory with exponent $H$ means
$$\mathrm{std}(S_n) \propto n^{H}, \qquad S_e(\omega) \propto |\omega|^{1-2H} \text{ as } \omega \to 0,$$
where $S_e(\omega)$ is the spectral density of the errors at angular frequency $\omega$. The second relation is the standard correspondence between fluctuation scaling and low-frequency power laws for stationary processes. For the Koopman operator, $S_e$ is the density of the spectral measure of the error observable, which is defined only on the continuous part of the spectrum. To give a sense of stakes, suppose $H = 0.8$. Then the 24-hour spread exceeds the random-walk prediction by a factor of $24^{0.3} \approx 2.6$. That gap is large enough to change battery sizing.
Why This Is Not Already Done
The closest work applies delay embedding and pseudospectral Koopman analysis to 1,494 European transmission stations. It reconstructs load from 100 eigenfunctions and runs 36 times faster than a tuned LSTM (Characterizing the load profile in power grids by Koopman mode decomposition of interconnected dynamics). That work motivates ours. It explicitly splits error into modal noise and innovation noise, but it never tests whether the innovation noise has memory, and it evaluates only on open-loop RMSE. Koopa handles non-stationarity with Fourier filtering and fits a local linear operator on each window. It also forces all dynamics into a finite matrix, so any continuous spectrum is folded into spurious discrete modes.
Two phase-space forecasters are the natural point-forecast engines here and represent the current strongest competitors. LETS Forecast (DeepEDM) shows that attention over delay vectors amounts to kernel regression on the reconstructed attractor. Attraos finds positive Lyapunov exponents in electricity benchmarks and concludes that Koopman methods are unsuitable for chaotic data. That conclusion constrains our design. We accept that the continuous, mixing part of the dynamics cannot be forecast pointwise. We ask the Koopman operator only for the second-order statistics of that part, and those are well defined precisely because the spectrum is continuous. Seen this way, the Attraos criticism applies to finite-matrix Koopman forecasters, not to spectral-measure estimation.
The estimation technique comes from the spectral-residual literature. ResKoopNet trains a neural dictionary of observables to minimize $\|\mathcal{K}\psi - \lambda\psi\|$ and so avoids spurious eigenvalues. It recovers the pendulum's continuous spectrum with 300 observables, where Residual DMD needs roughly 1,000. That paper is restricted to deterministic systems, a limit we must work around. We use residuals only to certify which eigenvalues are genuine calendar modes. We then estimate the continuous measure of the error observable from correlations, which remain meaningful for stationary stochastic signals. Resolvent-based treatments of continuous spectra (Generalized Eigenvalues of the Perron-Frobenius Operators of Symbolic Dynamical Systems; pseudoeigenfunctions from continuous spectra) supply the smoothing kernels we use to turn finite data into a density estimate. Detrended fluctuation analysis has so far served as a descriptive diagnostic in finance, neuroscience and ecology (Chaos and order in the bitcoin market; Scaling and memory in the non-Poisson process of limit order cancelation). Its known weakness, bias when several seasonal periods corrupt the log-log fit, is the problem our discrete/continuous split addresses. Finally, the downstream motivation comes from CommonPower. Its "forecaster paradox", in which a lower-MSE forecaster produces higher closed-loop cost because of unhedged underestimation, suggests that the structure of the errors matters as much as their size. No paper in the file examines the temporal memory of those errors.
Research Program
The first stage builds the spectral separation. We delay-embed each load series, including multi-station groups where available. We choose the delay by mutual information and check the embedding dimension with false-nearest-neighbour tests, and we report sensitivity across a grid because the choice is known to be fragile. We then train a residual-certified dictionary in the style of ResKoopNet. Eigenvalues with small residuals near the daily, weekly and annual frequencies are accepted as point spectrum. Their projection is removed from both the load and the forecaster's errors. The remainder is the continuous-spectrum component.
The second stage estimates the exponent in three ways, which should agree if the theory is right. These are DFA-2 applied to the continuous-component errors, a log-log fit of the smoothed spectral-measure density near zero frequency, and a direct variance-of-sums estimate of $\mathrm{std}(S_n)$. The state-conditional version uses a cross-spectral measure between the error observable weighted by a soft indicator of a region $A$ of the reconstructed attractor and the unweighted error observable. This equals the autocorrelation of errors given that the window starts in $A$. Its advantage over sliding-window DFA is that it pools all past visits to similar states, not just contiguous recent hours. Long-memory estimates need long records, and a sliding window cannot adapt quickly and still have enough data.
The third stage turns the estimate into margins. For an $n$-hour battery plan starting in state region $A$, the state-of-charge tube is $c_{\alpha}\,\sigma_A\, n^{H(A)}$. Here $\sigma_A$ is the one-step error scale in region $A$, and $c_\alpha$ is a quantile factor fitted on held-out data from a fractional Gaussian reference with matching $H$. The same tube, mapped through reserve requirements, sets up- and down-reserves asymmetrically, as the power-systems literature motivates (Youwei Jia et al.; Yuning Zhu et al.).
Baselines
The naive reference is the point forecaster with random-walk margins. The best available point forecaster produces forecasts, and its margins scale as $\sqrt{n}$ times the empirical one-step error. This is what an operator implicitly assumes when treating errors as white. CommonPower's safeguard layer is kept active so that violations show up as safeguard interventions and not as physical failures.
The strongest point forecasters are DeepEDM and Attraos, since both work on the reconstructed attractor and so share our embedding. DeepEDM outperforms iTransformer, PatchTST and Koopa in 36 of 48 multivariate settings across a suite that includes ECL (LETS Forecast). No per-dataset MSE value for ECL is reported on Lacuna. Attraos reaches competitive accuracy with 0.2M parameters against PatchTST's 2.4M. For residential data we add TSMixer with the event-response plug-in, which lowers MAE by 11.83% on average across Ampds2, UK-Dale and UMass (From Dense to Sparse). Each forecaster is retrained with the same lookback, data splits and hyperparameter budget of 30 trials per dataset. Our method never replaces these forecasters. It changes only the margins around their output, so any cost gain cannot come from a better point forecast.
The strongest uncertainty baselines are split and adaptive conformal intervals calibrated directly on $S_n$ for each $n$, and quantile regression on cumulative targets with the pinball loss. Both are tuned on the same validation data. For closed-loop cost on the Kerber Landnetz 30-bus benchmark, no reported monetary figure or violation rate is found on Lacuna for CommonPower. We therefore rerun its LSTM and Transformer configurations ourselves as the closed-loop reference.
Three ablations isolate the key ingredients. Global-$H$ margins drop state conditioning. STL-plus-DFA margins replace the Koopman separation with ordinary seasonal decomposition. Direct conditional-autocorrelation margins estimate the state-conditional exponent by averaging lagged error products over visits to region $A$, with no spectral smoothing. Our method counts as beating the field if, at matched average operating cost, it lowers the rate of state-of-charge safeguard interventions below every competitor's. Equivalently, at matched intervention rate it must lower cost, with non-overlapping bootstrap intervals across at least three load sources.
Why not just…?
Why not just fit a fractional noise model (ARFIMA) to the residuals of DeepEDM and size margins from its global $H$? This is a real threat, and it is exactly the global-$H$ ablation. If the error memory is uniform across the grid's operating states, this two-stage recipe captures everything that matters, and the Koopman machinery adds only a cleaner seasonal removal. Our advantage depends on $H$ varying with state, for example higher during sustained weather-driven regimes than in ordinary weekday cycles. The experiment tests this directly: the between-region spread of $H(A)$ must exceed its bootstrap uncertainty. If it does not, we will report that the global model suffices.
Why not just calibrate conformal intervals on the cumulative error? Conformal prediction needs no model of memory. Its guarantee rests on exchangeability, which long-range dependence violates, and it provides marginal coverage rather than coverage conditional on state. We have no cited evidence on how badly coverage degrades for load errors, so we measure it. Our measure is coverage of $S_n$ for $n$ from 1 to 48 hours, split by attractor region and by the Real-E-style high-volatility periods (Real-E). If adaptive conformal matches our conditional coverage, it wins on simplicity and the operational claim falls.
Why not just remove seasonality with STL or a Fourier filter and then run DFA? Multiple seasonal periods and their harmonics bias the log-log fit, a known limitation of DFA. A Fourier filter also discards the smeared power next to each calendar peak that a continuous spectrum puts there. We test this on synthetic series with known answers: fractional Gaussian noise of known $H$, plus daily and weekly cycles with drifting amplitude, plus a chaotic Lorenz observable. We then compare the bias of each estimator of $H$. If STL-plus-DFA is as accurate, the Koopman stage should be dropped and the proposal reduces to its empirical and operational claims.
Why not just train a more accurate or cost-aware point forecaster? CommonPower already shows that point accuracy does not settle cost. Even a perfect cost-aware point forecast still leaves residual errors whose running sum drives the battery. Our margins sit on top of any forecaster, so this alternative complements the proposal and is included by construction.
Decisive Checks
The first check can end the project early. We fit the best forecasters on ECL, the four ETT subsets, Re-Europe and the residential datasets, then estimate $H$ of their continuous-component errors with all three estimators. If $H$ is statistically indistinguishable from 0.5 at horizons of 6 to 72 hours, the long-memory story is false for load. We will publish that negative result, which would itself vindicate the white-innovation assumption of the prior Koopman grid work. The second check is agreement between the three estimators and recovery of the true exponent on the synthetic suite, which validates the spectral-measure route. The third is the variance of $H(A)$ across attractor regions against its bootstrap spread. The fourth is closed loop. The Kerber Landnetz grid is driven by Simbench profiles and, for transfer, by residential and Real-E demand traces rescaled to the feeder. Batteries are dispatched by CommonPower's model-predictive controller under each margin rule, and we report safeguard interventions, state-of-charge violations before projection, and operating cost. The prediction that sharpens the whole proposal is that the violation-rate advantage over random-walk margins grows with the planning horizon roughly as $n^{H-0.5}$. If the gap stays flat with horizon, then whatever helps is not long memory.
References
- Characterizing the load profile in power grids by Koopman mode decomposition of interconnected dynamics
- Koopa: Learning Non-stationary Time Series Dynamics with Koopman Predictors
- LETS Forecast: Learning Embedology for Time Series Forecasting
- Attractor Memory for Long-Term Time Series Forecasting: A Chaos Perspective
- ResKoopNet: Learning Koopman Representations for Complex Dynamics with Spectral Residuals
- Generalized eigenvalues of the Perron-Frobenius operators of symbolic dynamical systems
- Enhancing spectral analysis in nonlinear dynamics with pseudoeigenfunctions from continuous spectra
- Chaos and order in the bitcoin market
- Scaling and memory in the non-Poisson process of limit order cancelation
- CommonPower: A Framework for Safe Data-Driven Smart Grid Control
- A Novel Retrospect-Inspired Regime for Microgrid Real-Time Energy Scheduling With Heterogeneous Sources
- Medium- and long-term power load forecasting model
- From Dense to Sparse: Event Response for Enhanced Residential Load Forecasting
- Real-E: A Foundation Benchmark for Advancing Robust and Generalizable Electricity Forecasting