Catching Entrenched Shortcuts in Streaming Forecasters

A forecasting model trained on a live data stream can pick up a coincidence. Two sensors trend together for a few months, or a nuisance channel happens to lead the target during one season, and the model starts using that co-movement as if it were a real dependency. Image classification already shows what can follow. A shortcut learned early becomes entrenched, and later data that contradicts it does not dislodge it: in Where is the Truth? The Risk of Getting Confounded in a Continual World, a model that keeps every past example but trains on them in arrival order reaches 72.6% on shortcut-free test data, against 95.7% for the same data shuffled together. Nobody has asked whether streaming forecasters fall into the same trap. The forecasting tools built for drifting data, which rescale each input window, do not help here, because a spurious co-trend survives rescaling intact.

We propose two things. The first is a way to measure, at every point in a stream, how much a forecaster relies on a given input channel: replace that channel's history with a surrogate that keeps its frequency content but breaks its alignment with the other channels, then see how much the forecast moves. The second is a rule for which past windows to replay. Replay only windows where the model does much worse than the window's intrinsic predictability says it should. The error of a simple linear predictor on the same window gives that predictability as a cheap lower bound. The central claim is that raw loss is the wrong replay signal in forecasting. A window can have high error for two reasons: it breaks an entrenched shortcut, or it is simply noisy. Only the first kind is worth replaying, and error in excess of predictability separates the two.

Core Claim

Continual confounding, meaning shortcuts that are predictive in one stretch of the stream and misleading afterward, should appear in multivariate forecasting streams. When it does, it should show up in three ways. Reliance on a transiently predictive channel should persist after the co-movement ends. Sequentially trained models with full history should stay measurably worse than models trained jointly on the same data. The standard remedy, rescaling each window by its own mean and variance, should not remove the effect.

The proposed replay criterion is excess error: model error on a window divided by the linear lower bound from Spectral Coherence Predictability. Spectral Coherence Predictability is a per-window measure of how much of the future is linearly recoverable from the past, computed from frequency-by-frequency coherence in Beyond Model Ranking: Predictability-Aligned Evaluation for Time Series Forecasting. That paper reports it correlates with realized error at $R \ge 0.8$ and costs under a millisecond per instance. We claim this criterion closes more of the gap between sequential and joint training than loss-based replay, and that the advantage grows as the stream contains more intrinsically noisy segments.

Let $f_t$ be the model at stream time $t$, $w$ a window with lookback $x$ and horizon $y$, and $\mathrm{MSE}_{\mathrm{lin}}(w)$ the linear error bound implied by spectral coherence on $w$. The replay priority is

$$ e(w) = \frac{\mathrm{MSE}(f_t(x), y)}{\mathrm{MSE}_{\mathrm{lin}}(w) + \epsilon}, $$

with $\epsilon$ a small constant for numerical stability. The reasoning is as follows. When a spurious channel decouples, the target is still predictable from its own history and its true drivers, so $\mathrm{MSE}_{\mathrm{lin}}$ stays low while the entrenched model's error rises, and $e$ is large. On a noisy window both terms rise together, so $e$ stays moderate. Loss-based binning, as in Towards Unbiased Continual Learning: Avoiding Forgetting in the Presence of Spurious Correlations, cannot tell these cases apart.

The reliance score for channel $j$ at time $t$ is

$$ R_j(t) = \mathbb{E}_w \frac{\lVert f_t(x) - f_t(\tilde{x}^{(j)}) \rVert}{\lVert f_t(x) \rVert}, $$

where $\tilde{x}^{(j)}$ replaces channel $j$'s lookback with a phase-randomized surrogate. A phase-randomized surrogate keeps the channel's power spectrum and destroys its timing relative to the other channels. This makes $R_j$ a forecasting counterpart of the bias-flip misclassification rate, which counts how often a classifier's prediction changes when a known spurious feature is flipped. That rate is used in Continual Learning in the Presence of Spurious Correlations: Analyses and a Simple Baseline. Unlike it, $R_j$ needs no segmentation masks or group labels. The static reliance metrics, namely the Grad-CAM neuron score of An XAI-based Analysis of Shortcut Learning in Neural Networks and the information-decomposition audit of Towards Formalizing Spuriousness of Biased Datasets Using Partial Information Decomposition, both require masks.

Relation to Prior Work

Three lines of work motivate the gap. First, the continual-learning studies show shortcut entrenchment and forward and backward bias transfer, but only for sequences of discrete image or text tasks with categorical confounders: Where is the Truth? and Lee et al.. We move the question to continuous forecasting streams, where there are no task boundaries or groups.

Second, What If We Let Forecasting Forget? A Sparse Bottleneck for Cross-Variable Dependencies documents that dense attention across channels latches onto coincidences in noise. Its sparse top-$K$ routing (MS-FLOW) lowers a synthetic "pollution hit rate" from 7.79% to 3.80%. That work is static and offline and never asks whether a learned coincidence can be unlearned later in a stream. We use it as a backbone and a competitor, not as the answer.

Third, A General Spatio-Temporal Backbone with Scalable Contextual Pattern Bank for Urban Continual Forecasting handles streaming forecasting with a frozen backbone and a growing bank of adaptable parameters. It reports a 21.93% error reduction on CA-Stream, but it treats all drift as legitimate. We test whether its adaptable bank also memorizes transient shortcuts.

The rescaling methods constrain the design without solving the problem. These are Reversible Instance Normalization, which rescales each window and inverts the rescaling on the output; Non-stationary Transformers, which reinjects the removed statistics into attention; and APT, which conditions the rescaling on calendar-time prototypes. All three act on the level and scale of each window. A co-trend between two channels is invariant to that operation, so we expect these methods to leave entrenchment untouched. APT may even add a new risk: calendar prototypes can latch onto holidays or seasons that happened to coincide with a confounder.

On the method side, the group-free replay idea comes from Learning without Shortcuts. We keep its core move, replaying the samples that break the shortcut, and replace its selection signal. ERIS disentangles domain and label features for time series classification across subjects, and R-MDN regresses out known confounders online. Both need domain or confounder labels, which forecasting streams lack.

Research Program

We first build a controlled benchmark for continual confounding in forecasting.

In the synthetic version, targets are generated by a known sparse linear-plus-nonlinear system. A nuisance channel is made to lead the target during an early segment and is then decoupled, either in "strict" mode (it becomes random) or "disjoint" mode (it disappears), mirroring the ConCon settings. Separately, we inject noise bursts with low predictability so that raw loss and excess error disagree.

In the semi-synthetic version, we take ETTh1, ETTm1, Weather, and ECL and append a nuisance channel built from the target's future plus noise during the first third of the stream, then from an unrelated series afterward. For real streams we use PEMS-Stream and AIR-Stream from the pattern-bank paper. AIR-Stream has strong temporal drift, and on it we look for naturally occurring transient co-movements using $R_j(t)$.

On this benchmark we run three analyses. The first establishes whether entrenchment exists. We compare sequential fine-tuning, cumulative training on all history, and joint training on post-break error, and we track $R_j(t)$ for the nuisance channel. The second compares replay selectors under a fixed buffer. The third addresses the open tension between deep entrenchment and last-layer fixes. An XAI-based Analysis of Shortcut Learning found that refitting only the output layer raised worst-group accuracy from 0.37 to 0.71 on ISIC while the internal spurious scores were unchanged, which suggests the good features survive internally. The continual-learning evidence suggests the opposite under sequential training. We test which holds for forecasters by refitting only the output projection on post-break data and checking whether both $R_j$ and error recover.

Baselines

The naive reference is plain sequential fine-tuning on the stream with the same backbone, and next to it reservoir replay, which samples the buffer uniformly at random with the same buffer size. Any method that fails to beat reservoir replay on post-break error has shown nothing.

Cumulative training on the full history is the "keep everything" competitor. Joint training on the shuffled full stream is an oracle upper bound. Neither is a deployable method. Together they define the gap we try to close, reported as the fraction of the cumulative-to-joint gap recovered at equal gradient steps.

The strongest existing replay competitor is Learning without Shortcuts. We adapt it to forecasting by binning windows on raw loss and keeping high-loss windows, and we drop its $k$-means reweighting because forecasting has no classes. Its reported worst-group accuracy is 58.84% against 21.79% for DER++ on Biased CelebA in Towards Unbiased Continual Learning. No reported result on Lacuna exists for it on forecasting streams.

The strongest architectural competitor is MS-FLOW, which outperforms iTransformer by 8.98% in MSE on 12 benchmarks according to What If We Let Forecasting Forget?. The strongest continual forecasting competitor is the pattern-bank model from the urban continual forecasting paper. No reported result on Lacuna exists for either under continual confounding.

We also run RevIN and APT wrappers around an iTransformer backbone, and output-layer refitting as the post-hoc competitor. The key ablation holds everything fixed and swaps the replay priority between raw loss, excess error $e(w)$, and an oracle that knows which windows break the shortcut. This isolates whether normalizing by predictability is what matters.

For fairness, every method uses the same backbones (iTransformer with RevIN, and MS-FLOW), the same buffer size in windows, the same number of gradient steps per stream segment, and the same hyperparameter search of 20 trials tuned on an early-stream validation segment. Beating a baseline means lower post-break MSE and lower $R_j$ on the nuisance channel, both significant across five seeds, without worse pre-break error.

Why not just…?

Why not just rescale each window with RevIN or APT? These methods remove level and scale drift. A spurious co-trend between two channels is unchanged by subtracting each channel's mean and dividing by its standard deviation. Over-rescaling also blurs the attention patterns a model needs to detect regime changes, as Non-stationary Transformers shows. Our wrapper baselines test this directly. If RevIN alone drives $R_j$ to zero after the break, entrenchment is really a scale artifact and our claim fails.

Why not just use a sparse cross-channel bottleneck like MS-FLOW? This is the most serious threat. Sparse routing reduces latching onto coincidences in offline training, but a channel that is genuinely predictive for months will win a top-$K$ slot and may stay there. We run the full comparison on MS-FLOW as a backbone. If MS-FLOW with reservoir replay matches our selector on post-break error, the contribution shrinks to the measurement tool alone, and we will report it that way.

Why not just retrain from scratch on all data periodically, or refit only the output layer? Full periodic retraining is joint training. It is our upper bound, and its cost grows with stream length, which is exactly what streaming methods try to avoid. We report the compute ratio. Output-layer refitting is cheap and worked in static image settings. If it recovers post-break error in forecasting, that answers the entrenchment-versus-readout question in favor of the readout, and replay selection becomes unnecessary. We treat that as a real possible outcome of the third analysis.

Why not just replay the most recent data or high-loss windows? Recency weighting forgets legitimate long-range structure, and the pre-break error check exposes that. Loss-based replay is exactly what the key ablation tests against. Its predicted failure is buffer pollution by noise bursts, which we vary deliberately in the synthetic streams.

Decisive Checks

The first check is whether the phenomenon exists at all. On the synthetic strict streams, cumulative training must be clearly worse than joint training after the break, and $R_j$ must stay elevated. If forecasters simply unlearn the coincidence, we report that negative result and stop there. It would itself be informative against the image-domain findings.

The second check is whether excess-error replay beats loss-based replay by a margin that grows with the density of noise bursts. If the two selectors tie when there is no noise and diverge as noise increases, the mechanism is confirmed.

The third check is whether $R_j(t)$ measured before the break predicts the size of the post-break error across runs, which would make it an early warning usable on real streams without labels.

The fourth check is whether, on AIR-Stream, the channels flagged by high transient $R_j$ correspond to co-movements that later break.