Error-Gated Consolidation: Letting the Shape of Surprise Decide What a Network Forgets

A model that learns from a never-ending stream faces two kinds of change. Sometimes the world drifts slowly, as when friction wears down or a robot's payload grows heavier over hours. Sometimes it switches abruptly, as when a system is swapped out or a new task begins. Each kind calls for a different response. Slow drift should be tracked and gradually written into long-term memory. An abrupt switch should leave long-term memory alone, get handled by fast weights, and be remembered in case the old regime returns. Current methods are built for one kind of change and break on the other. Methods that watch for spikes in prediction error ("surprise") and treat them as task boundaries miss smooth drift. Methods that spread each weight across several variables that change at different speeds handle drift well. But after an abrupt switch, their slow variables drag the network back toward the stale regime.

The proposal is to use prediction error measured over several time windows to control how fast knowledge moves between the fast and slow parts of a multi-timescale weight memory. A short, sharp burst of error, or a large error sustained over a medium window, cuts the link between fast and slow variables at the matching depth. The slow knowledge is then protected and saved as a snapshot. Small, steadily tracked error leaves the links open, so drift consolidates normally. Put in two sentences: the network keeps its weights at several speeds, as in synaptic consolidation, and the temporal profile of its own surprise decides which speeds are allowed to absorb a change. The claim is that this one gating rule matches boundary-detection methods on switching streams and matches consolidation methods on drifting streams. It should beat both, and beat a naive stitching-together of the two, on streams that contain both kinds of change.

Where the Gap Is

The two nearest pieces of prior work split the problem between them. CoSFan forecasts physical systems with a hypernetwork, which infers the current system from recent frames. It detects regime changes from spikes in its own prediction error and uses those detections to organize a replay buffer, clustered with a Gaussian mixture, so that past regimes are not forgotten. It models dynamics at a single timescale, and it assumes the world is stationary between switches. That assumption is exactly where continuous drift defeats a thresholded spike detector.

Fast and Slow Successor Features represents each parameter as a chain of variables $u_1,\dots,u_K$. The first variable is plastic and each later one is slower. Applying this chain to predictive representations lets reinforcement-learning agents survive smooth mass and slip-probability drift, where EWC and plasticity-injection methods fail. It has no notion of an event boundary. Its authors also report that it must be trained with plain SGD, because Adam's per-parameter scaling disrupts the chain.

Several other papers motivate the gating idea without supplying it. Lifelong Neural Predictive Coding uses spikes in prediction-error variance to route learning into separate sparse subnetworks, on static image classification with no temporal hierarchy. Blocked training facilitates learning of multiple schemas shows that large errors at block transitions make a Bayesian latent-cause model split its representations, while a flat LSTM suffers catastrophic interference. Hierarchical Multiscale RNNs let learned boundaries control when higher layers update, but only in activations and without any study of forgetting. On the neuroscience side, partially nested cortical hierarchies of neural states and neural states that track stable environmental features indicate that slower cortical levels change state at event boundaries whose timing depends on the input, not on a fixed clock. Those rigid clocks are what Clockwork VAEs hard-code.

What no paper in this set does is apply boundary gating to the weight-consolidation chain itself, and do it per level, using error integrated over windows matched to each level's timescale.

Core Mechanism

Take a multi-timescale chain for every trainable weight, as in SF+SC. Only the fast variable $u_1$ receives gradient updates. After each optimizer step, the chain relaxes toward agreement:

$$u_k \leftarrow u_k + \frac{\alpha}{C_k}\Big[g_{k-1}(u_{k-1}-u_k) - g_k(u_k-u_{k+1})\Big], \qquad g_k = \sigma\!\big(-\beta\,(z_k-\theta)\big).$$

In this update, $C_k$ is the "capacity" of level $k$ and grows geometrically with depth, which makes deeper levels slower. $\alpha$ is a step size, and $g_k\in[0,1]$ is the gate on the link between levels $k$ and $k+1$. The gate is computed from $z_k$, a normalized score of recent prediction error. Specifically, $z_k$ is the forecast error averaged over a trailing window of length $\tau_k$, minus its running mean, divided by its running standard deviation. The window $\tau_k$ is matched to level $k$'s relaxation time. $\sigma$ is the logistic function, with a sharpness $\beta$ and a threshold $\theta$.

The gates behave differently for the two kinds of change. Under drift, the running statistics follow the slowly rising error, so every $z_k$ stays near zero, all gates stay open, and the chain behaves exactly like SF+SC. Under an abrupt switch, the short- and medium-window scores jump. The gates at those depths close, which does two things at once. The fast levels are no longer pulled back toward stale slow values, and the slow levels are no longer overwritten by the transient.

When a gate at depth $k$ closes, the model saves a snapshot of the levels at and below that depth into a small library. It also scores each stored snapshot by how well it predicts the recent window, and loads one if it beats the current state by a margin. This handles regimes that come back. If the new regime persists, its error settles, the running statistics absorb the change, and the gates reopen. The new regime is then consolidated. A deep snapshot is created only when a surprise lasts long enough to register at a deep window, which is the proposal's answer to the open problem of how coarse a given error spike is.

Two engineering choices are hypotheses that the experiments will test. First, the chain relaxation is kept as a separate deterministic step after the optimizer update. The aim is for Adam to see only $u_1$, which may avoid the incompatibility SF+SC reports; whether it actually does is an open question. Second, the error statistics are recomputed at each window scale and exclude samples collected while a gate was closed. This addresses the contamination problem noted for the online boundary discriminator in ESTimator, where a large boundary error stays in the rolling queue and distorts the threshold for the next boundary.

Research Program

The main testbed is continual latent-dynamics forecasting on CoSFan's gravity-6 and mixed-physics streams, rebuilt in three versions. The pure-switch version reproduces the original piecewise-stationary streams, with recurring regimes. The pure-drift version holds one system fixed and moves its physical parameter continuously, using the sine, non-periodic and Ornstein–Uhlenbeck schedules that SF+SC applied in Miniworld. Examples are the gravity angle in gravity-6 and the spring constant or pendulum length in mixed-physics. The drift-and-switch version combines both: parameters drift within each block and systems switch abruptly between blocks. The mixed versions are constructed for this proposal from the existing simulators; they are not published benchmarks.

The backbone for forecasting is CoSFan's latent dynamics model, so that the only thing changing across methods is how weights are updated and protected. Evaluation reports four things. Forecasting quality uses CoSFan's distance metric (DST). Backward transfer is re-measured on every past regime at the end of the stream. Recovery time is the number of steps after a switch until error returns to within a fixed band of its pre-switch level. Drift-tracking error is the mean error during drift segments. Because the switch times are known in simulation, the gate firings can also be scored as boundary detections, with precision and recall. The false-trigger rate on pure-drift streams deserves special attention.

The second testbed is control. A Dreamer-style world model and policy are run on the DeepMind Control Walker. Body-mass drift, as in SF+SC, is added on top of the abrupt Walk→Uphill→Downhill→Nofoot sequence from LLWM. Neither task identities nor switch times are given to the agent.

A small side study checks the granularity claim on real video. A causal next-feature predictor is run over Kinetics-GEBD, whose boundaries are annotated as shot changes (19.0%) or changes of action (63.6%). The test asks whether short-window error scores fire mainly at shot changes and longer-window scores mainly at action changes. The side study says nothing about forgetting. It only checks whether error measured over different windows separates boundaries of different depth, which is the premise the gating rule depends on.

Baselines

The naive reference point is a single set of weights fine-tuned online with Adam on the stream, with no protection against forgetting. Next to it is a reservoir replay buffer of fixed size and EWC. CoSFan already showed that reservoir sampling breaks when regimes are imbalanced or recur. S-NCN reports EWC falling to about 19% single-head accuracy on Split MNIST. Both are cheap and familiar, so they belong in the table.

The strongest method on abrupt switches is CoSFan itself, with its error-spike trigger and Gaussian-mixture replay, run through its released design on the same backbone. The strongest method on drift is the SF+SC consolidation chain, with the same number of levels and capacities as the proposal but no gates, trained with SGD as its authors specify. I found no reported scalar on Lacuna for CoSFan's DST on mixed-physics or gravity-6. The file says only that it achieved "substantially lower" DST and near-zero backward-transfer interference. I also found no reported scalar for SF+SC on Miniworld or DMC drift. Both baselines will therefore be reproduced, and all comparisons will use reproduced numbers.

For control, the reference numbers in the file come from LLWM. On Continual DMC it reports an average reward of 812, against 450 for standard DreamerV2. LLWM uses task identities to freeze and switch its world-model snapshots, so it serves as a task-oracle upper reference. Beating the flat baseline decisively, and approaching LLWM's figure without task labels, would count as success.

The most important baseline is the naive composition: CoSFan's single-threshold spike detector feeding a hard reset of the SF+SC chain, plus snapshot storage. Its ingredients are all off the shelf, and a skeptic would try it first. Two ablations isolate the key ingredient. The single-window ablation computes every gate from one shared error window. The no-library ablation keeps the gates but drops snapshot storage and retrieval. An oracle-boundary variant, which receives the true switch times, bounds how much detection errors cost.

Fairness rules are fixed in advance. Every method uses the same backbone, the same stream seeds, and the same memory budget, where snapshot storage is counted against the replay buffer size. Each method also gets the same hyperparameter search, a fixed number of configurations tuned on a held-out stream seed with a different regime order.

The criterion for beating the baselines is specific. On drift-and-switch streams, the proposal must beat both specialists and the naive composition on final DST and backward transfer, with non-overlapping confidence intervals across seeds. It must also stay within noise of CoSFan on pure-switch streams and of SF+SC on pure-drift streams.

Why not just…?

Why not just bolt a change-point detector onto the consolidation chain and reset it at detections? This is the real threat, and it is why the naive composition is a headline baseline rather than a footnote. The proposal predicts that this composition fails in two measurable ways. Under drift, a single fixed threshold either misses slow change or fires falsely as error creeps up; CoSFan's own stationarity assumption points to this weakness. And a hard reset throws away slow-level knowledge that the current regime could have reused. The drift-and-switch streams and the false-trigger rate on pure drift will settle this. If the composition matches the gated chain on those streams, the multi-window claim is wrong, and the contribution shrinks to an engineering recipe.

Why not just use a longer consolidation chain, with more and slower levels, and no gates? SF+SC's success under drift is real. But its authors only tested smooth change, and nothing in the chain can tell a transient apart from a new regime. The proposal predicts that after an abrupt switch the ungated chain shows long recovery times, because slow levels pull the fast weights back toward the old regime. Recovery time on pure-switch and mixed streams, measured as a function of chain length, directly tests this. If longer ungated chains recover as fast as gated ones, the gates are unnecessary.

Why not just use one adaptive threshold on a single error window, as ESTimator does? ESTimator reaches an average F1 of 0.748 for online boundary detection on Kinetics-GEBD, so single-window adaptive thresholds clearly work for finding boundaries. The issue here is different: deciding how deep a change goes. A single window cannot distinguish a brief glitch from a persistent regime change from slow drift. The single-window ablation on the physics streams, and the shot-change versus action-change analysis on Kinetics-GEBD, test whether multiple windows add information beyond the one window that ESTimator uses.

Why not just rely on in-context adaptation, a model that infers the current regime from recent frames, and skip weight protection? CoSFan's hypernetwork already adapts to a new system in one forward pass. Even so, its authors needed boundary-triggered, clustered replay to avoid forgetting, and the flat LSTM of Beukers et al. suffered catastrophic interference across schemas. For regimes that recur often, strong in-context inference may genuinely suffice. The comparison with CoSFan on pure-switch streams with frequent recurrence is where the proposal could lose. That outcome would be reported as marking where gating is not needed.

Decisive Checks

The idea stands or falls on one pattern: parity on each specialist's home stream, and a clear win on the mixed stream over both specialists and the naive composition. Three results would refute it. The first is if the single-window ablation performs as well as the full multi-window gating, which would mean error depth carries no useful signal. The second is if gated chains fire falsely on pure-drift streams at rates comparable to the fixed-threshold detector, which would mean the running normalization does not do its job. The third is if the relaxation step decoupled from Adam still degrades the chain relative to SGD, which would mean the optimizer restriction reported for SF+SC is not avoided and the method inherits SF+SC's slower training.

A positive Kinetics-GEBD side result, in which longer windows align with action changes more than with shot changes, would support the premise that the timescale of surprise reflects the depth of the change. A null side result would not refute the forgetting claims, but it would weaken the story connecting the gates to event hierarchies in the brain.