A sensor network installed in spring has no record of what summer looks like. Any model trained on its first months will be deployed into seasons whose average behavior and noise level it has never seen. Energy demand models trained on autumn and winter miss summer cooling peaks. Tropospheric delay models trained on one year degrade in the next (Global, spatially explicit modelling of zenith wet delay with XGBoost). The standard remedy is to keep learning after deployment with an online corrector, but every online method with a guarantee needs to be told the scale of the coming noise, either as a step size, a variance bound, or a confidence radius. Seasonal scarcity is exactly the situation in which that number is unknown, because the noisiest season has not happened yet.
This note proposes an online forecast-correction algorithm whose step sizes and prediction intervals are both computed from one quantity: the squared prediction errors the algorithm has already made, each weighted by how novel its input was. Online learning theory has a recipe for turning abstract worst-case guarantees into concrete algorithms, called admissible relaxations (Relax and Randomize : From Value to Algorithms). We use that recipe, but replace its worst-case scale terms with this self-normalizing quantity. The resulting algorithm runs in closed form at a few matrix operations per step. It does not need to imagine future seasons, and it produces anytime-valid intervals whose width tracks the season the system is actually in.
Core Claim
Two existing results meet here. Relax and Randomize : From Value to Algorithms supplies the algorithmic method. It replaces the intractable value of the online prediction game with a computable upper bound, the relaxation, and predicts by minimizing that bound one step at a time. When the bound cannot be written in closed form, it is estimated by "random playouts" that simulate the future from a known worst-case distribution. That route assumes we can specify what the future looks like, which is precisely what a single-season training record cannot provide.
On Equivalence of Martingale Tail Bounds and Deterministic Regret Inequalities supplies the statistical lens. It proves that pathwise regret inequalities and martingale concentration bounds are two faces of the same statement. In Hilbert-like spaces, the resulting tail bound scales with the realized variation $\sum_t \|d_t\|^2$ of the process, not with the horizon. Here $d_t$ is the increment of the process at step $t$. This is self-normalization: dividing a statistic by its own observed fluctuation so that unknown scale factors cancel. The proof of that equivalence is non-constructive.
Our claim is that, for online linear correction of a base forecaster under square loss, a relaxation built from this realized variation is admissible and computable in closed form. The algorithm derived from it achieves regret of the form
$$\mathrm{Regret}_T \;\le\; C \sum_{t=1}^{T} r_t^2\, z_t^\top V_t^{-1} z_t \;+\; \lambda \|\theta^\ast\|^2 .$$
The symbols are as follows. $z_t \in \mathbb{R}^d$ is the correction feature vector at time $t$ (base-model forecast, recent residuals, a few harmonics). $V_t = \lambda I + \sum_{s \le t} z_s z_s^\top$ is the regularized design matrix. $r_t$ is the realized residual at step $t$. $\theta^\ast$ is the best fixed correction in hindsight, and $C$ is an absolute constant.
The bound charges each step only for its own error, weighted by how new its input was. A calm winter therefore costs little, a turbulent summer costs more, and neither cost must be known in advance. The second product reads the same accumulated quantity, through the equivalence theorem, as the studentizing denominator of a confidence statement. That statement takes the place of the fixed sub-Gaussian radius $R$ in the confidence ellipsoid of Online Least Squares Estimation with Self-Normalized Processes: An Application to Bandit Problems. The aim is to obtain both the forecaster and its intervals from one object, at $O(d^2)$ cost per step, without playouts or oracles.
The form of this bound is the self-normalized analogue of the classical online ridge regret, in which a fixed $Y^2$ multiplies $\log\det V_T$. Replacing $Y^2$ with realized residuals at each step is what self-normalization means here. The open technical question is whether this can be done without knowing the residual range in advance. The "square root gap" flagged in On Equivalence of Martingale Tail Bounds and Deterministic Regret Inequalities and the logarithmic cost of constructivity noted in Bridging the Gap Between Minimax Value and Algorithmic Efficiency both suggest we may lose a logarithmic factor. We will report that loss rather than hide it.
What Is and Is Not New
The self-normalization literature already cancels unknown scales along the trajectory of an online estimator. Online Local Differential Private Quantile Inference via Self-normalization and its federated extension Federated Learning of Quantile Inference under Local Differential Privacy remove an unknown density from quantile confidence intervals with $O(1)$ memory. Both assume a stationary target and an asymptotic limit. We take their trajectory-studentization idea into an adversarial, non-stationary regime with finite-time guarantees.
Online Least Squares Estimation with Self-Normalized Processes: An Application to Bandit Problems self-normalizes the geometry of the inputs through $V_t$, but keeps the noise scale $R$ fixed and known. Under seasonal heteroscedasticity, a fixed $R$ is either too wide in winter or invalid in summer. We self-normalize the noise scale as well.
Generalization bounds for non-stationary mixing processes handles drift with a discrepancy term between past and target distributions. That term cannot be estimated from a record that never visited the target season. Our guarantee is worst-case over sequences, so it needs no discrepancy estimate. The price is that it is a regret statement about a correction layer, not a generalization statement about the base model.
Finally, Online Non-Parametric Regression shows that square loss permits fast rates through offset complexity, a version of sequential Rademacher complexity that subtracts a quadratic term. Its forecasters still need a smoothness index and empirical suprema. We restrict to a linear correction class precisely so that the offset relaxation becomes a closed-form expression.
Sequential Rademacher complexity therefore plays a specific role in this proposal. It is the quantity our relaxation upper-bounds at every step, conditioned on the observed history. That complexity measures how well a function class can fit random signs along every branch of a tree of possible futures. The tree is never enumerated; self-normalization lets us bound it using only the branch actually taken.
Research Program
The theoretical part has three steps.
The first step writes the conditional offset sequential Rademacher complexity of the linear correction class given history $h_t$. It then upper-bounds this complexity by $-\min_\theta \big[\sum_{s\le t}(y_s-\theta^\top z_s)^2+\lambda\|\theta\|^2\big]$ plus an accumulated term $\sum_{s \le t} \hat r_s^2\, z_s^\top V_s^{-1} z_s$. The bound is completed by a correction for the residual range, learned online by a doubling-style restart.
The second step proves the admissibility inequality of Relax and Randomize : From Value to Algorithms for this relaxation and extracts the minimizing prediction. We expect it to be a clipped online ridge prediction with an adaptively scaled regularizer.
The third step reads the same accumulated term, via the equivalence theorem, as a self-normalized tail bound. This yields anytime-valid intervals of the form
$$|y_{t+1}-\hat y_{t+1}| \;\le\; \sqrt{\beta_t(\delta)}\;\sqrt{\textstyle\sum_{s\le t} \hat r_s^2\, z_s^\top V_s^{-1} z_s \,/\, t}\;\sqrt{1+z_{t+1}^\top V_t^{-1} z_{t+1}},$$
where $\beta_t(\delta)$ is a logarithmic confidence factor at level $1-\delta$.
A known risk is that a single global normalizer averages over seasons and lags at a transition. We will therefore also analyze a phase-indexed variant. It keeps a separate accumulated term for each of $K$ calendar phases, which raises memory to $O(Kd^2)$. Seasons not yet seen start empty and fall back to the global term.
The empirical part wraps this layer around fixed base forecasters trained on deliberately truncated records, then runs it online through the remaining years.
Baselines
The naive reference points are persistence and the frozen base model with no online correction. Persistence's 3-day 500 hPa geopotential RMSE on WeatherBench is reported as "~936 m^2s^-2" in WeatherBench: A Benchmark Data Set for Data-Driven Weather Forecasting. In the same paper, the direct CNN is reported at "~626 m^2s^-2", linear regression at "~693 m^2s^-2", and the operational physical model at "~154 m^2s^-2". Those numbers come from full multi-decade training. They serve here as calibration of how much a one-year training record costs, not as targets. The direct CNN will be retrained on our truncated split as one base model, using the same architecture and recipe.
On Weather2K, the strongest reported base model is MFMGCN, which "consistently achieved the lowest" MAE and RMSE according to Weather2K: A Multivariate Spatio-Temporal Benchmark Dataset for Meteorological Forecasting Based on Real-Time Observation Data from Ground Weather Stations. However, no reported numerical result was found on Lacuna for its MAE or RMSE. We will retrain it on the truncated split. We will also use SimVPv2 through the OpenSTL Benchmark Suite, since OpenSTL: A Comprehensive Benchmark of Spatio-Temporal Predictive Learning found recurrent-free models consistently better on WeatherBench at a fraction of the compute.
The strongest competitors for the correction layer itself all use the same features $z_t$, the same base forecasts, and the same online stream. The first is online ridge regression with the Online Least Squares Estimation with Self-Normalized Processes: An Application to Bandit Problems ellipsoid, with $R$ tuned on the training season; this is what a careful practitioner would deploy. The second is the same estimator with $R$ replaced by a rolling-window residual standard deviation, with the window length tuned on the training season. The third is online gradient descent with an adaptive, scale-free step size computed from accumulated squared gradients. The fourth is an adaptive online conformal interval wrapped around online ridge. Every method gets the same tuning budget, a grid search on a held-out slice of the training season only, because touching unseen seasons would defeat the purpose.
The key ablation keeps our algorithm intact but replaces the self-normalizing term with its horizon-scaled worst-case counterpart. This isolates whether pathwise normalization, rather than the ridge structure, drives any gain. A second ablation compares the global and phase-indexed normalizers.
Beating the baselines means two things. Our method must match or beat tuned-$R$ online ridge on point RMSE in unseen seasons. It must also hold nominal interval coverage within one percentage point in every calendar month, with narrower average width than the rolling-window and conformal intervals. Point-forecast parity combined with month-wise valid and tighter intervals is the result that justifies the method.
Why not just…?
Why not just standardize residuals with a rolling-window variance and plug it into the usual ellipsoid? The plug-in radius is only asymptotically valid. Online Least Squares Estimation with Self-Normalized Processes: An Application to Bandit Problems built its self-normalized sets precisely to be valid at every finite $t$, without a $t\to\infty$ argument. A rolling window also faces a trade-off at season onset: short windows are noisy and long windows lag. The month-wise coverage experiment below measures that lag directly. If a tuned rolling window holds coverage at transitions, the interval half of our claim collapses and we will say so.
Why not just run adaptive-step online gradient descent? For point forecasts, this is a real threat. Scale-free adaptive gradient methods already obtain regret in terms of realized squared gradients, and they may match our RMSE. Our method differs in two ways. It exploits the curvature of square loss through the offset relaxation, which Online Non-Parametric Regression shows is what gives logarithmic rather than square-root regret. It also yields intervals from the same quantity, whereas gradient methods provide none. If adaptive gradient descent ties on RMSE, the contribution reduces to the intervals and the curvature-aware analysis, and we will present it that way.
Why not just use online conformal prediction for the intervals? Adaptive conformal methods give long-run average coverage without distributional assumptions, so they are the strongest interval competitor. They achieve average coverage by over-covering calm months to compensate for under-covering turbulent ones. Their width also does not shrink with input novelty the way $z^\top V^{-1} z$ does. Our test reports coverage per calendar month and per inter-annual anomaly year, not pooled. If conformal methods are month-wise valid at equal width, our interval claim fails.
Why not just add day-of-year features to the base model? Harmonic features capture the seasonal mean only for seasons present in training. They cannot learn summer's noise level from a winter record. Bridging the Temporal Generalization Gap in Predictive Models and the inter-annual degradation in Global, spatially explicit modelling of zenith wet delay with XGBoost show that calendar structure does not transfer across years. We include harmonics in $z_t$ for every method, so this alternative is absorbed into all comparisons rather than competing with them.
Proposed Experiments
On Weather2K-S (15 stations, hourly), base models are trained only on January–April 2017. They are then deployed online, at 1-, 6- and 12-hour horizons, for temperature, humidity and visibility through August 2021. That stream crosses every unseen summer and four inter-annual cycles.
On WeatherBench at 5.625°, a CNN and SimVPv2 are trained on a single year. The corrector then runs per grid cell on 2-meter temperature and 500 hPa geopotential through the following years. This mirrors the single-year protocol under which Global, spatially explicit modelling of zenith wet delay with XGBoost observed degradation.
On Caravan, daily streamflow in strongly seasonal catchments supplies heavy-tailed, sharply heteroscedastic residuals. Training uses one hydrological season. This is the hardest test for a normalizer that assumes 2-smooth geometry.
The metrics are RMSE and MAE by calendar month; empirical coverage and mean width of 90% intervals by month and by year; cumulative regret against the best fixed correction in hindsight; and wall-clock cost per step. Two checks decide the outcome. The first is the coverage-versus-month curve across the first unseen summer: the self-normalized intervals must stay within one point of nominal while the tuned-$R$ ellipsoid either under-covers or is uniformly wide. The second is the horizon-scaled ablation, which must lose width or coverage; otherwise self-normalization is not doing the work. Regret should grow with $\sum_t r_t^2 z_t^\top V_t^{-1} z_t$ rather than with $T$, which we check by plotting the two against each other across stations. We will also report the measured logarithmic penalty relative to an oracle that knows each season's noise scale in advance.
References
- Global, spatially explicit modelling of zenith wet delay with XGBoost
- Relax and Randomize : From Value to Algorithms
- On Equivalence of Martingale Tail Bounds and Deterministic Regret Inequalities
- Online Least Squares Estimation with Self-Normalized Processes: An Application to Bandit Problems
- Multiclass Transductive Online Learning
- Online Local Differential Private Quantile Inference via Self-normalization
- Federated Learning of Quantile Inference under Local Differential Privacy
- Generalization bounds for non-stationary mixing processes
- Online Non-Parametric Regression
- WeatherBench: A Benchmark Data Set for Data-Driven Weather Forecasting
- Weather2K: A Multivariate Spatio-Temporal Benchmark Dataset for Meteorological Forecasting Based on Real-Time Observation Data from Ground Weather Stations
- OpenSTL: A Comprehensive Benchmark of Spatio-Temporal Predictive Learning
- Adaptive Automatic Target Recognition with SVM Boosting for Outlier Detection