When a human learns to ride a bicycle, they do not suddenly forget how to walk. In contrast, standard artificial neural networks suffer from catastrophic forgetting: when trained on a new task, the gradient updates required to minimize the new loss function often overwrite the weight configurations essential for previous tasks.
As Ian Goodfellow and Yoshua Bengio demonstrated in An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks, this isn't a minor glitch but a fundamental consequence of how we train models. This challenge is often framed as the stability-plasticity dilemma: a model must be "plastic" enough to integrate new knowledge but "stable" enough to prevent the erasure of existing representations.
The Palimpsest Problem
In early neuromorphic research, Srinjoy Mitra and colleagues referred to this as the "palimpsest problem"—named after ancient manuscripts where new text was written over partially erased old text. In a neural network, the "ink" is the set of synaptic weights. Because a single set of weights is typically used to solve all tasks, any update is a zero-sum game.
To solve this, researchers have converged on three primary strategies: replaying the past, regularizing the present, and expanding the future.
Replaying the Past
The most intuitive way to prevent forgetting is to remind the model of what it once knew. This is known as Experience Replay (ER). Instead of training only on new data, the model maintains a small "memory" of representative samples from previous tasks.
- Selective Memory: Fan Zhou and Chengtai Cao applied this to graph structures in ER-GNN, showing that even a tiny subset of "representative nodes" can anchor a model's performance.
- Efficiency through Saliency: Storing full images is memory-intensive. In Saliency Guided Experience Packing for Replay in Continual Learning, Kaushik Roy suggests "packing" only the most informative patches of an image, allowing the model to store a more diverse history in the same memory footprint.
- Generative Reproduction: When data cannot be stored due to privacy or scale, models can use "pseudo-rehearsal." Chen He and Ruiping Wang explored using generative models to "imagine" old data, which is then used to interleave with new training samples.
Regularizing the Present
If we know which weights are important for Task A, we can penalize the model for changing them while it learns Task B. This "elastic" approach allows the model to learn new things using less critical parameters.
Hongjoon Ahn introduced Uncertainty-regularized Continual Learning (UCL), which uses the model's own confidence to decide which weights to freeze. Similarly, Marcin Mazur and colleagues developed CW-TaLaR, which uses the Cramer-Wold distance to ensure that the distribution of features in the network's internal layers doesn't drift too far from the original task's distribution.
Expanding the Future
Rather than forcing a fixed-size network to learn everything, some architectures grow as they encounter new information.
- Dual Memories: Sang-Woo Lee proposed a Dual Memory Architecture that mimics the brain's hippocampus and cortex: a "Fast Memory" for rapid adaptation to new streams and a "Deep Memory" for long-term consolidation.
- Modular Stacking: In HC-Net, Jangho Kim and Nojun Kwak "stack" new convolutional filters alongside frozen old ones. This ensures that the "detectors" for Task 1 remain physically untouched while Task 2 gets its own dedicated hardware.
- Parameter-Efficient Tuning: In modern Vision Transformers, Massimiliano Mancini showed that simply tuning the LayerNorm parameters for each task—rather than the whole model—can prevent forgetting with minimal overhead.
Beyond Not Forgetting
While most research focuses on preventing a performance drop, the ultimate goal of the field is forward and backward transfer. As Joshua T. Vogelstein argues in Simple Lifelong Learning Machines, a truly intelligent system shouldn't just "not forget" Task A; learning Task B should actually make it better at Task A. By treating tasks as independent representations that are ensembled together, we move from a model that merely survives new data to one that thrives on it.