When a neural network learns a new task, it typically updates its weights to minimize error on the new data. However, in a sequential learning setting, these updates often overwrite the parameters essential for previous tasks, leading to a total loss of prior knowledge—a phenomenon known as catastrophic forgetting.
To solve this, researchers face the stability-plasticity dilemma: a model must be "plastic" enough to acquire new information but "stable" enough to prevent the interference of new gradients with old memories.
The Cost of Stability: Model Intransigence
The most common way to enforce stability is through regularization. Techniques like Elastic Weight Consolidation (EWC), explored by Ben Glocker and Konstantinos Kamnitsas in medical imaging, identify which weights are "important" for past tasks and penalize changes to them.
However, this creates a secondary problem: model intransigence. As Denis Jered McInerney and Byron C Wallace observe, if regularization is too aggressive, the network becomes too "stiff" to learn anything new. The model is so focused on staying close to its previous state that it cannot adapt to the unique characteristics of the current task. This tension is particularly visible in generative models, where Kamil Deja and Tomasz Trzcinski found that simple weight penalties often lead to a collapse in the quality of generated outputs.
Granular Control: Beyond Global Penalties
To move past the limitations of global weight penalties, researchers have developed more surgical methods of control:
- Neuron-Level Control: Instead of looking at connection weights, Inyoung Paik and Injung Kim 0001 proposed Neuron-level Plasticity Control (NPC). By dynamically adjusting the learning rates of individual neurons based on their importance to past tasks, they achieve a more efficient balance than traditional weight consolidation.
- Sparse Allocation: Rather than using the whole network for every task, Yijun Yang and Tianyi Zhou developed CoTASP, which uses sparse "prompts" or masks to carve out specific sub-networks for each task. This allows similar tasks to share knowledge while physically isolating the parameters of distinct tasks to prevent overwriting.
- Structural Expansion: Jaehong Yoon and Sung Ju Hwang introduced Dynamically Expandable Networks (DEN), which automatically decide when to retrain existing neurons and when to "split" or add new ones to accommodate new information without drifting away from old concepts.
The Pre-training Paradigm Shift
A major realization in recent years is that the "best" continual learning algorithms change depending on where the model starts. Yu-Xiong Wang and Kuan-Ying Lee demonstrated that starting with Pre-trained Models (PTMs) completely reshuffles the leaderboard of algorithms.
In PTMs, the challenge isn't just preventing forgetting; it's leveraging the rich, existing feature representations. James Seale Smith and Junjiao Tian showed that for PTMs, even simple L2 regularization can outperform complex state-of-the-art methods because the model already possesses the necessary "vocabulary" to understand new tasks. The goal shifts from learning from scratch to adapting without destroying.
The Risk of Entrenchment
Stability has a hidden danger: it can protect bad habits. Kai Gu introduced the Einstellung Rigidity Index (ERI) to diagnose when a model is "stuck." He found that mechanisms designed to prevent forgetting—like replay buffers or strict regularization—can inadvertently entrench suboptimal "shortcuts" or features learned early in training. This rigidity prevents the model from discarding irrelevant information, effectively blocking it from learning more robust, task-specific features later on.
Ultimately, the field is moving away from seeing stability and plasticity as a zero-sum game. Instead, through methods like Information Maximization (Lama Alssum) and Mutual Information Maximization (Ziqi Gu), researchers are seeking objectives that guide the model to remain consistent with its past while remaining flexible enough to find optimal solutions for the future.