Remembering Rare Climate Regimes in Sub-Seasonal Bias Correction

The forecasts that work best 2 to 6 weeks ahead are not big neural networks. They are physics-based weather models whose systematic errors are removed by simple statistical corrections that are refit continually on a recent window of data. That window is how these corrections keep up with model upgrades and a warming baseline. It is also how they forget. Each time the window slides forward, it drops the last strong El Niño, or the last time the tropical rainfall pulse that drives much sub-seasonal weather (the Madden–Julian Oscillation, MJO) sat in a particular phase. When that regime returns, the correction has no memory of how the weather model misbehaved under it.

We propose to treat this as a catastrophic-forgetting problem, meaning a learner that loses old knowledge because it is updated only on recent data. Our fix keeps the correction simple but splits it into two parts. The first part tracks drift and is meant to be forgotten. The second part stores regime-specific error and is meant to be kept, and it is learned from the whole historical record once the drift has been removed. The correction is then evaluated where forgetting should hurt most: target dates whose climate state has not occurred inside the recent window.

Core Claim

Rolling-window bias correction mixes up two kinds of non-stationarity. One is slow, irreversible drift, such as model version changes and trend. The other is recurring, reversible regime dependence, such as ENSO (the El Niño–Southern Oscillation) and MJO phase. Recency weighting handles the first and destroys the second. Pooling all history keeps the second but is contaminated by the first. If the drift is estimated locally in time and subtracted from every historical date, the remaining errors can be pooled across decades and keyed by the continuous state of the climate drivers. This should give a correction that matches rolling-window skill on ordinary dates and beats it on rare-regime dates.

The idea is borrowed from continual learning but deliberately kept low-capacity. The borrowed parts are an additive split into shared and specialised parameters, regime membership inferred without task labels, and retroactive re-estimation of old parameters. The low capacity is a design constraint, because the sub-seasonal literature shows that expressive models overfit climate noise.

Where this sits in prior work

Two papers motivate the work by establishing that recency-adaptive correction is the method to beat, and by showing where it forgets. Adaptive Bias Correction for Improved Subseasonal Forecasting combines three components refit on rolling windows of recent lagged data: dynamical-model offsets, adaptive climatology, and persistence regressions. SubseasonalClimateUSA and Learned Benchmarks for Subseasonal Forecasting supply the contiguous-U.S. benchmark, the same family of corrections, and an AdaHedge online ensemble over them. Neither line has any mechanism for keeping regime-specific knowledge once it leaves the window. We keep their components and add a memory beside them.

Bridging the "Predictability Desert" extends this style of correction to full predictive distributions through quantile-wise debiasing that is refit daily, and it is the current competition winner. We treat it as a second testbed rather than a competitor to replace.

Two papers constrain the design. The benchmark papers report that deep sequence models such as Informer and N-BEATS lose to these linear corrections. That rules out putting a large neural continual learner in the loop, so our regime memory is a shrunk kernel regression and not a network.

Replay can provably increase forgetting proves that replaying a few old samples can turn vanishing forgetting into constant forgetting when task subspaces are close. It also shows that forgetting is non-monotonic in buffer size. Seasonal and ENSO regimes share the same grid and physics, so the most obvious memory, splicing a handful of past El Niño dates back into the rolling window, is exactly the design this result warns against. We test it directly instead of assuming it works.

Two papers supply the method. Additive Parameter Decomposition writes task weights as a shared base, a task mask, and a sparse task residual. It also updates old residuals retroactively when the base moves. We use the same additive split and the retroactive step, but APD assumes discrete, known task boundaries. Our regimes are instead soft weights computed from continuous driver indices, which addresses its stated task-boundary limitation.

The diversity-based exemplar selection of RBF k-DPP replay provides a stronger replay baseline. We build it on driver-index space so the comparison with our memory is fair.

We set aside the neural-architecture route. One version would continually train a model like TianQuan-S2S, a vision-transformer forecaster built on separate models per 5-day lead. Another would rely on the forgetting resistance of vision transformers reported by Qu et al.. Both would be testing forgetting inside a model class that does not yet win on this problem.

The model

For grid cell $s$ and issue date $t$, let $f_{s,t}$ be the raw dynamical forecast of a week 3–4 or 5–6 average. Let $z_t$ be the vector of driver indices known at issue time: a Niño-3.4 sea-surface temperature anomaly, the two MJO phase-space components, and optionally a polar-vortex index. The corrected forecast is

$$\hat y_{s,t} = f_{s,t} + d_s(t) + \sum_{k=1}^{K} w_k(z_t)\, r_{s,k}.$$

Here $d_s(t)$ is a drift offset fit on a rolling window ending at $t$, exactly as in the ABC dynamical-model correction. The $w_k(z_t)$ are normalised radial-basis weights centred on $K$ anchor points in driver space. The $r_{s,k}$ are regime residuals.

The key step is how the $r_{s,k}$ are fit. For every historical date $\tau$ in the full hindcast record, we compute the drift-removed error $e_{s,\tau} = y_{s,\tau} - f_{s,\tau} - d_s(\tau)$. Here $d_s(\tau)$ is the rolling-window estimate that would have been available at $\tau$. We then fit $r$ by weighted ridge regression of $e$ on $w(z_\tau)$. The ridge penalty shrinks the regime residuals toward zero, so in the limit of pure noise the method reduces to ABC.

Each new verification updates $d$ immediately. It also triggers a cheap retroactive refit of $r$, because changing the drift estimate changes the historical residuals. This is APD's retroactive update made trivial by linearity.

The same structure transfers to the probabilistic setting. The drift term becomes the quantile-wise debiasing that is refit daily, and the regime residuals become quantile-wise shifts, followed by the isotonic projection used in that paper.

Research Program

The first stage asks whether the premise is true. Before building anything, we run ABC in its published streaming hindcast protocol on SubseasonalClimateUSA. We then stratify target dates by how unusual their driver state $z_t$ is relative to the training window used at that date, measured as the kernel density of $z_t$ under the window's own driver states. We plot the skill of the correction minus the skill of the raw dynamical model as a function of this rarity score.

Our forgetting hypothesis predicts that ABC's gain over raw shrinks, or turns negative, at high rarity. If the curve is flat, rolling windows are not forgetting anything that matters, and the project stops here with a useful negative result.

The second stage builds the drift-plus-regime correction above. It also builds a ladder of memory mechanisms under an identical streaming protocol: no memory, random replay of past dates into the window at several buffer sizes, k-DPP replay on driver space, regime-conditioned analog windows, and our decomposition.

The third stage tests generality. We port the decomposition into the quantile-correction pipeline on ERA5 targets (ECMWF's global reanalysis, its best estimate of past weather). This checks whether the regime memory adds to probabilistic skill, or whether daily quantile refitting already captures it.

Baselines

The naive reference is the raw dynamical forecast, CFSv2 or ECMWF, with no correction, plus the standard operational lead-dependent climatological debiasing. It anchors every skill number.

The strongest existing method on this benchmark is ABC itself. For ECMWF, Adaptive Bias Correction reports anomaly-correlation skill improvements of 60–90% for temperature over baseline skills of 0.18–0.25, and 40–69% for precipitation over baseline skills of 0.11–0.15. SubseasonalClimateUSA reports ABC as 10% more accurate in RMSE and 329% more skillful in anomaly correlation than CFSv2. We found no reported result on Lacuna for ABC skill stratified by ENSO or MJO regime rarity, so the regime-stratified ABC numbers will be produced by this project.

We rerun ABC with its published window lengths and component set on the same targets, years and dynamical models. Our method inherits ABC's drift component unchanged, so any difference comes from the regime memory. Beating ABC means two things. First, a statistically significant gain in anomaly correlation on the top rarity tercile, under a block bootstrap over issue years. Second, no significant loss on the bottom two terciles or overall.

The AdaHedge ensemble from the same paper is the strongest decision-level competitor. We give it a regime-specialist ABC (described below) as an additional expert so it has a chance to route to regime knowledge.

A pooled static correction fit once on all history, with no drift term, tests whether simply remembering everything suffices.

Two ablations isolate the key ingredient. The first removes drift subtraction before fitting $r$, which tests whether separating drift from regime matters. The second replaces the radial-basis regime weights with a single linear term in each driver index. This tests whether a simple linear dependence on the drivers is enough.

Every learned variant gets the same hyperparameter search budget: window length, $K$, kernel width and ridge penalty. Tuning uses years before the evaluation period only.

Why not just…?

Why not just use a longer window, or pool all 40 years? Pooling keeps the rare regimes but also keeps biases from old model versions and an old climate baseline. The benchmark papers adopted rolling windows precisely because recency helps. The pooled static baseline and the no-drift-subtraction ablation measure this directly. If pooling matches us on rare dates without losing on common ones, the drift/regime split is unnecessary, and we will say so.

Why not just pick training dates by climate-state similarity instead of recency? This analog-window approach is the most serious threat. It is close in spirit to ABC's adaptive climatology, which already outperformed ECMWF on precipitation. Its weakness, in our hypothesis, is that a 1997 El Niño analog carries 1997-era model bias and background climate. Analogs therefore fix regime forgetting by reintroducing drift contamination. The analog baseline is run with the same tuning budget. If it matches the decomposition on rare-regime dates, the honest conclusion is that analog selection is the right fix, and our contribution narrows to the diagnosis in stage one.

Why not just replay a few stored El Niño dates into the rolling window? Replay can provably increase forgetting predicts that small buffers can hurt when regimes share structure. We sweep buffer size and expect to see either the non-monotonic curve that result predicts or a clean monotone benefit. Either outcome is informative for operational practice. The k-DPP variant tests whether diverse selection rescues replay.

Why not just add Niño-3.4 and MJO indices as regressors inside ABC? Inside a rolling window, a regressor that barely varies cannot be estimated, and the strong El Niño is exactly what the window lacks. Fit on all history instead, the regressor absorbs drift. The linear-index ablation, fit both ways, shows which failure dominates. If the linear version fit on drift-removed residuals does as well as the kernel version, we adopt it as the simpler method.

Proposed Experiments

The primary testbed is SubseasonalClimateUSA. The targets are contiguous-U.S. temperature and precipitation at weeks 3–4 and 5–6, from the CFSv2 and ECMWF forecasts it distributes. Evaluation follows the papers' streaming hindcast protocol over the most recent held-out years, using RMSE and anomaly correlation skill.

The forgetting-specific metric is skill as a function of driver-state rarity relative to the training window. A related metric is skill on regime recurrences as a function of years since that regime's last occurrence.

The secondary testbed is the global probabilistic setting with ERA5 targets, scored the way the quantile-correction paper scores it.

The decisive checks are three.

First, stage one must show that ABC's advantage over raw forecasts degrades with rarity. Otherwise the problem is not real.

Second, the decomposition must beat both ABC and the analog-window baseline on high-rarity dates without losing on ordinary ones. It should do so in both temperature and precipitation, and the drift-subtraction ablation must account for the gain.

Third, in the probabilistic port, the regime memory must improve skill over daily quantile refitting on rare-regime dates. Otherwise the method is not needed once corrections act on full distributions.

One limitation must be stated in advance. Reforecast records cover only a few decades, so strong-ENSO samples are few. The block bootstrap over years, and reporting the count of rare-regime dates alongside every number, are how we avoid overclaiming from a small number of events.