Mitigating Sequential Parameter Interference in Continual Learning

36 concepts36 papers1 proposal

When a standard artificial neural network learns a sequence of tasks, it suffers from a fundamental flaw: learning new information typically erases what was previously mastered. This phenomenon, known as catastrophic forgetting or sequential parameter interference, occurs because standard optimization algorithms like gradient descent update weights globally to minimize the current loss. In doing so, they blindly overwrite the specific parameter configurations required to maintain performance on prior tasks. As noted by Jin et al. in their work on memory editing, and emphasized by Nekoei et al. in non-stationary reinforcement learning environments, adaptive optimizers often exacerbate this by making large updates that aggressively overwrite past strategies.

To build models capable of lifelong learning, researchers must design mechanisms that protect past knowledge while leaving the network flexible enough to absorb new skills. Modern approaches solve this problem through three primary lenses: geometric subspace shielding, parameter isolation, and functional regularization.

Geometric Alignment and Subspace Shielding

One way to prevent weight overwriting is to restrict parameter updates to directions that do not interfere with past tasks. If we can identify the specific "subspace" or geometric structure a task occupies, we can shield it from future gradients.

He et al. implement this concept using "conceptors"—soft projection matrices that map out the activation subspaces utilized by previous tasks. During backpropagation, gradients are projected to avoid these regions, ensuring that updates for new tasks occur orthogonal to the pathways needed for old ones. Similarly, Roy et al. propose modeling feature distributions as low-dimensional subspaces on a Grassmannian manifold. By distilling these geometric structures rather than raw weights, the network preserves the underlying relationships of past data representations even as the backbone evolves.

Standard Gradient Update:
New Task Gradient ───> [ Overwrites Past Parameters ] ───> Forgetting

Subspace Shielded Update:
New Task Gradient ───> [ Projection Matrix / Conceptor ] ───> Safe Parameter Space
                                    │
                       (Blocks Intersecting Paths)

When weight-space regularization is used instead of explicit projection, its success depends heavily on how the parameter space is structured. For instance, Elastic Weight Consolidation (EWC) penalizes changes to important weights based on the Fisher Information Matrix, but it assumes this matrix is diagonal for computational feasibility. To make this assumption actually hold, Liu et al. introduce a method that rotates the network's layers. This reparameterization aligns the parameter space so that the Fisher Information Matrix becomes approximately diagonal, drastically improving the accuracy of weight consolidation.

Parameter Isolation and Modular Subnetworks

Instead of forcing a single set of shared weights to accommodate every task, parameter isolation strategies dedicate specific parameters to specific tasks, eliminating interference by design.

One approach isolates task-specific pathways within a single dense network. Kang et al. leverage the Lottery Ticket Hypothesis to identify and freeze "winning subnetworks" for each task, compressing them using Huffman coding to keep memory growth sub-linear. Similarly, the framework designed by Tu et al. freezes old weights and uses learnable masks to "pick" relevant knowledge for new tasks, growing the network only when capacity is saturated. To protect these boundaries without explicit task labels during inference, Kim et al. use hard attention masks combined with out-of-distribution (OOD) detection to automatically route inputs to the correct subnetwork.

Another variation of this strategy expands the model architecture externally by building libraries of modular experts or parameter-efficient adapters:

  • Expert Routing: Aljundi et al. introduce a network of specialized expert models paired with lightweight gating autoencoders that automatically route test samples to the appropriate expert.
  • Low-Rank Libraries: Lei et al. deploy a growing library of low-rank adapters managed by a lightweight router to blend skills dynamically for robotic tasks.
  • Retrievable Skills: Lee et al. isolate behaviors into modular, retrievable adapters, allowing robots to share knowledge across tasks and even cleanly "unlearn" specific behaviors without degrading other capabilities.

Even in newer architectures like Kolmogorov-Arnold Networks (KANs), which possess inherent local plasticity, parameter isolation remains necessary. Lee et al. demonstrate that KANs still suffer from catastrophic forgetting on complex tasks, but can be rescued by specialized parameter isolation algorithms that lock down critical activation functions.

Functional Regularization and Representation Replay

Rather than constraining the internal weights of a network, functional regularization focuses on preserving the network's outputs or latent representations. This provides the model with more optimization flexibility; weights can change freely as long as the overall input-output mappings remain stable.

Rudner et al. formalize this via Sequential Function-Space Variational Inference (S-FSVI). By regularizing predictions directly in the function space, the model avoids the overly restrictive constraints of weight-space methods, allowing parameters to find alternative configurations that satisfy both old and new tasks.

Weight-Space Constraint:  [W_old] ─── (Must stay exactly here) ───> Rigid / Low Plasticity
Function-Space Constraint: [W_new] ─── (Can move, if f(X) remains stable) ───> Flexible / High Plasticity

When past data cannot be stored due to privacy or memory constraints, models can replay synthetic representations to maintain functional stability. Ye et al. combine Variational Autoencoders with GANs to generate structured, disentangled latent representations of past domains, ensuring downstream tasks can be rehearsed without historical data. In class-incremental settings, Kanagarajah et al. use a feature "hallucinator" to generate synthetic vectors, ensuring that previously learned decision boundaries remain valid even after the underlying backbone updates.

For explicit knowledge preservation across tasks, distillation techniques are widely used. Chuang et al. employ a "disposable teacher" for each new language task, distilling specialized knowledge into a single generalist lifelong learner. In image restoration, Xiao et al. use a multi-teacher distillation strategy paired with Fourier-domain priors to learn how to remove new adverse weather effects sequentially without losing the ability to clear old ones.

Optimization Landscapes and Pre-training Geometry

The geometry of the loss landscape plays a decisive role in how vulnerable a model is to parameter interference. If a model settles into a sharp, narrow minimum for Task A, even a tiny shift in weights to accommodate Task B will cause a massive spike in Task A's loss. However, if the minimum is wide and flat, the weights can shift slightly without destroying past performance.

Sharp Minimum (High Forgetting):           Flat Minimum (Low Forgetting):
       \       /                                  \                 /
        \  *  /  <-- Small weight shift            \       *       /  <-- Small weight shift
         \___/       causes huge loss jump          \_____________/       leaves loss unchanged

Mehta et al. demonstrate that large-scale pre-training fundamentally alters this landscape geometry. Pre-trained models naturally bias optimization toward flatter minima, an effect that can be explicitly enhanced using Sharpness-Aware Minimization (SAM) to build inherent resistance to forgetting.

Taking this to the extreme, some frameworks eliminate weight updates within the backbone entirely during incremental learning. McDonnell shows that by freezing a pre-trained backbone and passing its features through high-dimensional random projections to a decorrelated class-prototype classifier, catastrophic forgetting can be completely bypassed while outperforming complex fine-tuning methods.

Alternative optimization paradigms also offer unique structural advantages:

  • Forward-Only Optimization: Feng et al. introduce ZeroFlow, demonstrating that zeroth-order optimization—which relies strictly on forward passes rather than backpropagation—mitigates sequential interference while drastically reducing memory overhead in restricted environments.
  • Analytical and Exemplar Learning: Gao et al. combine momentum-based adapter updates with a feature "rumination" mechanism to balance plasticity and stability. When exemplars are available, Iscen et al. show that using a lightweight transformer to explicitly compare new queries against a memory bank during inference preserves representation boundaries better than standard rehearsal.
  • Graph Replay: For non-Euclidean data structures, Zhou et al. adapt experience replay to Graph Neural Networks by identifying and storing a minimal subset of highly influential topological nodes to anchor the graph structure across sequential tasks.

Ultimately, the field is moving away from hand-crafted mitigation rules. As demonstrated by Irie et al., metalearning can be used to train self-referential neural networks that discover their own in-context algorithms for balancing stability and plasticity, automating the protection of parameters against sequential interference.

Key Foundational Work

Rotate your Networks: Better Weight Consolidation and Less Catastrophic Forgetting (2018) — contributed 1 concept elements to this direction.
Expert Gate: Lifelong Learning with a Network of Experts (2017) — contributed 1 concept elements to this direction.
Overcoming Catastrophic Interference using Conceptor-Aided Backpropagation (2018) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_catastrophic_forgetting_sequential_inter_9e06e065,
  title = {catastrophic forgetting (sequential interference)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/catastrophic-forgetting-sequential-interference/rdr_9e06e065a63d4530ac04c7f3c724cd34},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_catastrophic_forgetting_sequential_inter_9e06e065_v1,
  title = {catastrophic forgetting (sequential interference)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/catastrophic-forgetting-sequential-interference/rdr_9e06e065a63d4530ac04c7f3c724cd34/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake