When a neural network learns a sequence of distinct tasks, it faces a fundamental tension known as the stability-plasticity dilemma. To learn a new task, the network must be plastic—its weights must change to minimize the new task's loss. However, to retain past knowledge, the network must be stable—its weights must remain unchanged where they are critical to previous tasks.
Standard gradient descent completely ignores this balance. Because it assumes data is independent and identically distributed (i.e., stationary), it updates parameters globally. When the data distribution shifts to a new task, the gradients push the parameters out of the optimal loss valleys of past tasks and into the valley of the current task. This results in catastrophic forgetting, where previously acquired capabilities are abruptly erased.
To resolve this dilemma, modern continual learning frameworks modify how gradients alter the parameter space. They achieve this through three core strategies: regularizing parameter updates based on importance, isolating tasks into distinct structural modules, or reconstructing past data distributions via generative replay and abstract prototypes.
Regularization and Parameter Importance
One way to balance stability and plasticity is to allow all weights to change, but penalize updates to weights that are deemed critical for past tasks. This transforms the optimization landscape by adding a quadratic penalty for moving away from historical parameter configurations.
Standard Loss Landscape: [Task B Loss Only] -> Overwrites Task A
Regularized Landscape: [Task B Loss] + λ ∑ (Importance_i * (θ_i - θ_old_i)²)
But how do we determine which weights are important without storing all past data? In Continual Learning Through Synaptic Intelligence, Friedemann Zenke et al. proposed tracking parameter importance online during training. By computing the running sum of the product of the gradient and the parameter increment, the method estimates how much each individual synapse contributes to the overall drop in the task's loss. If a weight significantly drives down the loss for Task A, its importance score is high, and subsequent tasks face a steep penalty if they attempt to alter it.
To scale this idea to complex meta-learning scenarios without incurring massive computational overhead, Tong Lin et al. developed an efficient approximation framework in Efficient Meta-Learning for Continual Learning with Taylor Expansion Approximation. Instead of tracking exact parameter trajectories, the authors use a first-order Taylor expansion to estimate parameter importance, enabling rapid closed-form meta-updates that prevent forgetting while remaining highly memory-efficient.
Rather than just penalizing weight changes, we can explicitly project gradients into a subspace where they do not interfere with past tasks. In Quantized Gradient Projection for Memory-Efficient Continual Learning, Dongjun Kim et al. constrain gradient updates to be orthogonal to the input subspaces of previous tasks. To overcome the massive memory footprint of storing these historical subspaces, the authors introduce an outlier-resistant quantization strategy that compresses the projection memory by up to 6.4× without sacrificing accuracy.
Biologically inspired variants also operate at the optimizer level. In Introducing principles of synaptic integration in the optimization of deep neural networks, Giorgia Dellaferrera et al. introduced an optimizer called GRAPES, which modulates error signals based on local weight distributions. By mimicking how biological networks integrate synaptic inputs, it naturally protects critical pathways from being overwritten during backpropagation.
Modular Isolation and Subnetworks
If sharing parameters across tasks causes interference, an alternative solution is to isolate parameters structurally. Instead of updating the entire network, we can allocate specific pathways or subnetworks to specific tasks.
The most direct form of this is finding and freezing task-specific subnetworks within a single dense model. In Forget-free Continual Learning with Winning Subnetworks, Haeyong Kang et al. leverage the Lottery Ticket Hypothesis to identify a sparse "winning subnetwork" for each task. Once a task is mastered, its subnetwork weights are frozen. Subsequent tasks can only exploit the remaining unallocated weights or build upon existing ones without altering them, completely eliminating backward interference.
Task 1: [W1] [W2] [W3] --> Freeze W1, W3 for Task 1
Task 2: [W1_frozen] [W2_updated] [W3_frozen] [W4_new]
To make modular networks more flexible, several approaches dynamically compose structures based on task relationships:
- Dynamic Composition: In BNS: Building Network Structures Dynamically for Continual Learning, Qi Qin et al. use reinforcement learning to dynamically construct optimal network architectures for each new task. By explicitly modeling task similarity, the framework routes information through shared modules when tasks are related (enabling forward transfer) and branches out into new structures when tasks are dissimilar (preventing forgetting).
- Task-Driven Priors: Similarly, Tom Veniat et al. demonstrate in Efficient Continual Learning with Modular Networks and Task-Driven Priors how a modular network can learn to select a subset of modules for each task using a structural prior, optimizing both parameter reuse and task isolation.
- Information Routing: In Task-Aware Information Routing from Common Representation Space in Lifelong Learning, Prashant Shivaram Bhat et al. introduce TAMiL, which routes features from a shared representation space through tiny, task-specific attention bottlenecks. This isolates task execution and mitigates the network's bias toward recently learned tasks.
- Layer-Wise Entropy: Rather than hard-coding modules, Vladimir Araujo et al. propose an adaptive approach in Entropy-based Stability-Plasticity for Lifelong Learning. They use side-branch layers to measure prediction uncertainty (entropy). If a layer exhibits low entropy, it signifies that the layer already possesses stable, general knowledge and should be frozen; high-entropy layers remain plastic to absorb new task-specific details.
This modular philosophy extends across diverse learning paradigms. In Memory-Integrated Reconfigurable Adapters: A Unified Framework for Settings with Multiple Tasks, Susmit Agrawal et al. use associative memory to dynamically mix task-specific adapter weights on a per-sample basis. For sequential decision-making, Jorge A. Mendez et al. demonstrate in Modular Lifelong Reinforcement Learning via Neural Composition how decomposing policies into reusable neural modules allows agents to master complex sequences of control tasks with zero catastrophic forgetting.
Generative Replay and Prototypical Spaces
A third paradigm avoids modifying the network's internal structure or gradients directly. Instead, it corrects the data distribution itself. If catastrophic forgetting is caused by the absence of past data, we can simulate an i.i.d. data stream by interleaving new data with representations of the past.
When storing raw historical data is impossible due to memory constraints or privacy regulations, generative models can act as an artificial memory buffer. In DDGR: Continual Learning with Deep Diffusion-based Generative Replay, Rui Gao et al. employ diffusion models to generate high-quality synthetic samples of past tasks. By pairing the generator with a classifier via a bi-directional instruction-operator, the system replays crisp, accurate historical data alongside new data, keeping past decision boundaries intact.
New Task Data ───┐
├───> Joint Training ───> Updated Model
Synthetic Replay ┘
(from Diffusion)
Alternatively, we can bypass raw data entirely by operating in abstract, invariant feature spaces:
- Prototypical Transfer: In privacy-sensitive environments like federated learning, Xin Gao et al. present a solution in FedProK: Trustworthy Federated Class-Incremental Learning via Prototypical Feature Knowledge Transfer. Instead of saving or generating images, the authors transfer and align abstract "prototypical features" across sequential tasks, ensuring the global model retains historical class structures without exposing user data.
- Hyperdimensional Mapping: Taking abstraction to its logical extreme, Michael Hersche et al. introduce C-FSCIL in Constrained Few-shot Class-incremental Learning. They use hyperdimensional computing to map new classes into "quasi-orthogonal" vectors within a high-dimensional space. Because these vectors are mathematically isolated by the geometry of high dimensions, new classes can be added with zero retraining of the core feature extractor, completely bypassing the stability-plasticity trade-off.
- Test-Time Adaptation: Forgetting can also be managed dynamically during inference. In Controlling Forgetting with Test-Time Data in Continual Learning, Rahaf Aljundi et al. introduce DoSAPP, a framework that uses unlabeled data encountered at test time to perform double smoothing on parameter projections. This allows vision-language models to adjust their decision boundaries on the fly, maintaining a stable baseline of historical knowledge without requiring an explicit replay buffer during training.
- On-Device Pruning: On resource-constrained edge devices, Biqing Duan et al. show in LODAP: On-Device Incremental Learning Via Lightweight Operations and Data Pruning that combining lightweight adapters with an aggressive data pruning strategy—which discards redundant, "easy" training samples—allows models to learn incrementally within a strict memory budget while preserving past performance.
Whether by calculating parameter importance, isolating modular pathways, or reconstructing past distributions, these diverse mechanisms share a singular goal: breaking the global nature of backpropagation to ensure that learning today does not mean forgetting yesterday.
Key Foundational Work
BNS: Building Network Structures Dynamically for Continual Learning (2021) — contributed 1 concept elements to this direction.
Efficient Continual Learning with Modular Networks and Task-Driven Priors (2021) — contributed 1 concept elements to this direction.
Learning Then, Learning Now, and Every Second in Between: Lifelong Learning With a Simulated Humanoid Robot (2021) — contributed 1 concept elements to this direction.