When a neural network learns a sequence of tasks, it faces a fundamental conflict known as the stability-plasticity dilemma. To absorb new information, the network must be plastic—its weights must change. However, to retain previously acquired knowledge, the network must be stable—its weights must remain fixed.
In standard deep learning, gradient descent updates weights globally to minimize loss on the current data stream. When the data distribution shifts, these updates overwrite the precise parameter configurations optimized for earlier tasks. This causes catastrophic forgetting, where performance on prior tasks abruptly collapses. As highlighted by Hongjoon Ahn et al. in their work on Uncertainty-based Continual Learning with Adaptive Regularization and Shixing Chen et al. in Coupled End-to-End Transfer Learning With Generalized Fisher Information, resolving this tension requires moving beyond naive weight updates. Even in distributed environments like federated learning, Anastasiia Usmanova et al. demonstrate that local adaptation inherently triggers this knowledge degradation (Federated Learning and catastrophic forgetting in pervasive computing: demonstration in HAR domain).
To build networks that learn continuously, researchers focus on three intuitive strategies: smoothing the geometry of the loss landscape, isolating parameter updates to specific modules, and anchoring optimization using past knowledge.
The Geometry of the Loss Landscape
To understand why networks forget, it helps to visualize the loss landscape—the mathematical surface mapping parameter values to model error.
If a network finds a solution for Task A in a narrow, sharp valley, a tiny shift in weights to accommodate Task B will push the model up the steep walls of the valley, causing a massive spike in Task A's loss. Conversely, if the network finds a solution in a wide, flat valley, the weights can move slightly to satisfy Task B without significantly degrading performance on Task A.
Sharp Minimum (High Forgetting) Flat Minimum (Low Forgetting)
\ / \ /
\ * / <-- Small weight shift \ * / <-- Small weight shift
\_ _/ causes high loss \___ _ _____/ leaves loss low
By forcing optimization toward these flatter regions, models become naturally robust to subsequent updates. Ang Bian et al. leverage this intuition in Make Continual Learning Stronger via C-Flat, combining zeroth-order and first-order flatness measures to ensure the model lands in highly stable valleys. Similarly, Danruo DENG et al. introduce Flattening Sharpness for Dynamic Gradient Projection Memory Benefits Continual Learning, demonstrating that explicitly smoothing the loss landscape allows dynamic gradient projections to shift weights safely without disrupting old memories.
Architectural Isolation and Decoupling
Instead of allowing gradients to alter every parameter in a network, a powerful alternative is to restrict where learning happens. By decoupling the architecture, we can preserve core representations while allowing targeted plasticity.
Decoupling Features from Classifiers
One approach freezes the feature extractor (the backbone) and only adapts the classification boundaries. For example, Chi Zhang et al. propose Few-Shot Incremental Learning With Continually Evolved Classifiers, using a frozen backbone alongside a graph attention network to update class boundaries dynamically without altering underlying features. In a similar vein, Da-Wei Zhou et al. explore forward-compatible representations in Forward Compatible Few-Shot Class-Incremental Learning, ensuring early feature spaces leave room for future classes.
Sub-network and Parameter Tuning
For large models like Vision Transformers (ViTs), updating the entire network is computationally expensive and destabilizing. The authors of On the Effectiveness of LayerNorm Tuning for Continual Learning in Vision Transformers (Massimiliano Mancini et al.) show that simply tuning the scale and bias of LayerNorm layers provides enough plasticity to master new tasks while keeping the rest of the transformer rigid and stable.
Other methods isolate specific units or modules for targeted tasks:
- Pattern Neurons: In urban time-series forecasting, Chengxin Wang et al. identify specialized "pattern neurons" that handle rare events. By isolating and fine-tuning only these neurons via their PN-Train method, they prevent updates for rare events from degrading everyday forecasting accuracy (Investigating Pattern Neurons in Urban Time Series Forecasting).
- Modular Expansion: Fu-Yun Wang et al. introduce independent modules for past and future tasks in BEEF: Bi-Compatible Class-Incremental Learning via Energy-Based Expansion and Fusion, eliminating interference entirely by fusing distinct energy-based prototypes.
- Prompting and Attention: Rather than modifying weights, Long Zhao et al. steer class prototypes using learnable task prompts (Steering Prototypes with Prompt-tuning for Rehearsal-free Continual Learning), while Zhen Wang et al. design an "inter-task attention" mechanism to manage sequential data streams (Continual Learning with Lifelong Vision Transformer).
This tension between resetting parameters to gain plasticity and keeping them to maintain stability is also visible in reinforcement learning. As Huaicheng Zhou et al. observe, resetting high-utility neurons to restore learning capacity immediately degrades performance, requiring careful distillation strategies like Sustainable Backup Propagation to balance the reset (Stay Hungry, Keep Learning: Sustainable Plasticity for Deep Reinforcement Learning).
Regularization, Anchoring, and Replay
A third paradigm allows weights to change but introduces mathematical forces that anchor the network to its past behavior.
Functional Regularization
Instead of protecting specific weights, functional regularization protects the outputs of the network. A classic example is Learning without Forgetting (LwF). As demonstrated by Guy Oren et al. in In Defense of the Learning Without Forgetting for Task Incremental Learning, using the predictions of the previous model as a soft distillation target during new training acts as a powerful anchor, preventing the output space from drifting.
In probabilistic settings, standard Variational Continual Learning often suffers because approximation errors compound over time. To fix this, Luckeciano Carvalho Melo et al. introduce Temporal-Difference Variational Continual Learning, which regularizes the network against a historical sequence of past posterior estimates rather than just the most recent one, stabilizing the optimization trajectory.
Generative and Feature Replay
When storing raw data from past tasks is impossible due to privacy or memory constraints, models can generate their own reminders. This mirrors biological brains, where the reactivation of past neural patterns during sleep consolidates memory, a concept reviewed by Giri P. Krishnan et al. in Replay in Deep Learning: Current Approaches and Missing Biological Elements.
In generative modeling, Fei Ye et al. implement this via a dual-memory system—an expanding "Teacher" ensemble that passes down long-term knowledge to a compact "Student" VAE (Continual Variational Autoencoder via Continual Generative Knowledge Distillation). For classification, Ruitao Wu et al. create a mutual boosting loop between a classifier and a diffusion model (Diffusion-Classifier Synergy: Reward-Aligned Learning via Mutual Boosting Loop for FSCIL). The diffusion model synthesizes strategically beneficial "hard samples" that reinforce old class boundaries, ensuring the classifier does not forget them.
Alternatively, models can store compact feature representations rather than raw images or videos. Adrian G. Bors et al. utilize a highly efficient feature-based memory buffer to manage unlabeled video streams (Unsupervised Video Class-Incremental Learning via Deep Embedded Clustering Management), while Linlan Huang et al. leverage the frozen semantic space of vision-language models alongside singular value decomposition (SVD) to fuse parameters without storing any historical data (Class-Incremental Learning with CLIP: Adaptive Representation Adjustment and Parameter Fusion).
Finally, alternative learning paradigms can bypass traditional backpropagation entirely to mitigate forgetting. Jangho Lee et al. show that brain-inspired Predictive Coding (PC), which minimizes error locally within layers rather than propagating a global error signal backward, naturally reduces parameter overwriting during incremental tasks (Brain-inspired Predictive Coding Improves the Performance of Machine Challenging Tasks).
Key Foundational Work
Uncertainty-based Continual Learning with Adaptive Regularization (2019) — contributed 1 concept elements to this direction.
Coupled End-to-End Transfer Learning With Generalized Fisher Information (2018) — contributed 1 concept elements to this direction.