Optimizing the Stability-Plasticity Tradeoff in Sequential Learning

This direction is split into other directions: catastrophic forgetting (stability-plasticity dilemma), stability-plasticity dilemma (catastrophic forgetting). Its last version stays readable and citable here.

Version 1 of 1448 papers453 conceptsfrozen 2026-10-04

When a human learns to ride a bicycle, they do not suddenly forget how to walk. In contrast, standard artificial neural networks suffer from catastrophic forgetting: when trained on a new task, the gradient updates required to minimize the new loss function often overwrite the weight configurations essential for previous tasks.

As Ian Goodfellow and Yoshua Bengio demonstrated in An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks, this isn't a minor glitch but a fundamental consequence of how we train models. This challenge is often framed as the stability-plasticity dilemma: a model must be "plastic" enough to integrate new knowledge but "stable" enough to prevent the erasure of existing representations.

The Palimpsest Problem

In early neuromorphic research, Srinjoy Mitra and colleagues referred to this as the "palimpsest problem"—named after ancient manuscripts where new text was written over partially erased old text. In a neural network, the "ink" is the set of synaptic weights. Because a single set of weights is typically used to solve all tasks, any update is a zero-sum game.

To solve this, researchers have converged on three primary strategies: replaying the past, regularizing the present, and expanding the future.

Replaying the Past

The most intuitive way to prevent forgetting is to remind the model of what it once knew. This is known as Experience Replay (ER). Instead of training only on new data, the model maintains a small "memory" of representative samples from previous tasks.

  • Selective Memory: Fan Zhou and Chengtai Cao applied this to graph structures in ER-GNN, showing that even a tiny subset of "representative nodes" can anchor a model's performance.
  • Efficiency through Saliency: Storing full images is memory-intensive. In Saliency Guided Experience Packing for Replay in Continual Learning, Kaushik Roy suggests "packing" only the most informative patches of an image, allowing the model to store a more diverse history in the same memory footprint.
  • Generative Reproduction: When data cannot be stored due to privacy or scale, models can use "pseudo-rehearsal." Chen He and Ruiping Wang explored using generative models to "imagine" old data, which is then used to interleave with new training samples.

Regularizing the Present

If we know which weights are important for Task A, we can penalize the model for changing them while it learns Task B. This "elastic" approach allows the model to learn new things using less critical parameters.

Hongjoon Ahn introduced Uncertainty-regularized Continual Learning (UCL), which uses the model's own confidence to decide which weights to freeze. Similarly, Marcin Mazur and colleagues developed CW-TaLaR, which uses the Cramer-Wold distance to ensure that the distribution of features in the network's internal layers doesn't drift too far from the original task's distribution.

Expanding the Future

Rather than forcing a fixed-size network to learn everything, some architectures grow as they encounter new information.

  • Dual Memories: Sang-Woo Lee proposed a Dual Memory Architecture that mimics the brain's hippocampus and cortex: a "Fast Memory" for rapid adaptation to new streams and a "Deep Memory" for long-term consolidation.
  • Modular Stacking: In HC-Net, Jangho Kim and Nojun Kwak "stack" new convolutional filters alongside frozen old ones. This ensures that the "detectors" for Task 1 remain physically untouched while Task 2 gets its own dedicated hardware.
  • Parameter-Efficient Tuning: In modern Vision Transformers, Massimiliano Mancini showed that simply tuning the LayerNorm parameters for each task—rather than the whole model—can prevent forgetting with minimal overhead.

Beyond Not Forgetting

While most research focuses on preventing a performance drop, the ultimate goal of the field is forward and backward transfer. As Joshua T. Vogelstein argues in Simple Lifelong Learning Machines, a truly intelligent system shouldn't just "not forget" Task A; learning Task B should actually make it better at Task A. By treating tasks as independent representations that are ensembled together, we move from a model that merely survives new data to one that thrives on it.

Papers in this version

Showing 60 of 448 papers.

History & citationVersion 1 · 2026-10-04

Split into other directions · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_optimizing_the_stability_plasticity_trad_df3f7d1c,
  title = {Optimizing the Stability-Plasticity Tradeoff in Sequential Learning},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/optimizing-the-stability-plasticity-tradeoff-in-sequential-learning/rdr_df3f7d1c686e4d038e95021d70aa14a4},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 1, frozen text):

@misc{lacuna_optimizing_the_stability_plasticity_trad_df3f7d1c_v1,
  title = {Optimizing the Stability-Plasticity Tradeoff in Sequential Learning},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/optimizing-the-stability-plasticity-tradeoff-in-sequential-learning/rdr_df3f7d1c686e4d038e95021d70aa14a4/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}