In continual learning, a model faces the stability-plasticity dilemma: it must be plastic enough to integrate new information but stable enough to prevent "catastrophic forgetting" of previous knowledge. Traditionally, researchers balanced these competing needs using fixed hyperparameters to weight distillation losses or regularization terms. However, recent evidence suggests that a static balance is insufficient for the complex, shifting distributions of real-world data.
Instead, a new generation of frameworks focuses on dynamic stability-plasticity balancing. These methods treat the importance of new and old knowledge not as a constant, but as a variable that should adapt to the composition of the data, the model’s current performance, and the relationship between tasks.
Compositional Scaling: The Ratio of New to Old
The most intuitive signal for balancing stability and plasticity is the relative size of the new information being introduced. If a model is learning a single new class alongside fifty old ones, the risk of overwriting the entire feature space is high; conversely, if the new task is massive, the model must prioritize plasticity to achieve any meaningful accuracy.
Hou et al. pioneered this by scaling distillation loss based on the ratio of new classes to old classes. This heuristic ensures that as the total number of learned concepts grows, the model adaptively increases the weight of knowledge preservation. Similarly, Xiao et al. and Zhang et al. applied this logic to weight fusion, where the parameters of an old model and a newly trained model are merged. By calculating fusion coefficients based on the class ratio, the authors ensure the updated model's influence is proportional to the amount of new information introduced.
In streaming environments where task boundaries are less clear, Mi et al. and Zhang et al. proposed adjusting weights based on the proportion of novel items within a specific training batch. This fine-grained adaptation allows the model to remain stable when new information is sparse and become plastic when a surge of new trends appears.
Feedback-Driven Adaptation: Learning from Performance
A more sophisticated approach moves beyond simple counting to look at how the model is actually performing during the update process. If the model is struggling to learn a new task (high loss), it needs more plasticity; if it has already mastered the new task (low loss), it should shift its focus back to stability to protect its memory.
The MEGA framework, developed by Guo et al., formalizes this as a mixed stochastic gradient problem. The authors dynamically adjust the weights of gradients from current and historical tasks based on real-time loss. When the current task's loss is low, the optimization prioritizes stability by focusing on episodic memory. Koh et al. take a similar approach using model confidence: high confidence in a prediction signals that the model can safely prioritize preserving existing knowledge, while low confidence triggers a need for adaptation.
Relational and Temporal Balancing
The difficulty of the stability-plasticity tradeoff also depends on what is being learned. If a new task is highly similar to an old one, the model can rely on existing features, requiring less plasticity.
Piao et al. introduced class-aware distillation that measures the correlation between new and old classes. When tasks are highly related, the system enforces stronger consistency to preserve shared features. For vision-language models, Zheng et al. proposed re-weighting the influence of "teacher" models based on the drift of individual samples, anchoring foundational knowledge when significant deviation is detected.
Finally, the balance can be managed temporally. Knights et al. utilized a sigmoid-based weighting scheme that starts with high stability to protect legacy knowledge and gradually relaxes constraints to allow for plasticity as the training of a new task progresses. As the number of tasks grows, Bao et al. argue that the importance of stability-focused losses must scale upward to compensate for the increasing difficulty of retaining a massive historical knowledge base.
Automating the Tradeoff
The shift toward dynamic balancing culminates in methods that treat hyperparameter tuning itself as an online learning problem. Liu et al. formulated class-incremental learning as a Markov Decision Process, using a bandit-based algorithm to automatically tune distillation weights and learning rates in response to the data stream. By moving away from fixed heuristics and toward adaptive, feedback-loop systems, these researchers are creating models that can navigate the shifting requirements of lifelong learning without manual intervention.
Key Foundational Work
ADER: Adaptively Distilled Exemplar Replay Towards Continual Learning for Session-based Recommendation (2020) — contributed 3 concept elements to this direction.
Improved Schemes for Episodic Memory-based Lifelong Learning (2020) — contributed 3 concept elements to this direction.
Learning a Unified Classifier Incrementally via Rebalancing. (2019) — contributed 1 concept elements to this direction.