In the study of Continual Learning (CL), the primary villain has traditionally been "catastrophic forgetting"—the tendency of neural networks to overwrite old weights when learning new information. However, recent research suggests that even if we perfectly preserve old knowledge, models still fail.
The problem is not just that the model forgets what a "dog" looks like; it's that the model never learned how to tell a "dog" (from Task 1) apart from a "cat" (from Task 2). This phenomenon, identified by researchers like Zixuan Ke and Bing Liu as Inter-Task Class Separation (ICS), represents a fundamental bottleneck in class-incremental learning.
The Hidden Hurdle: Cross-Task Discrimination
Most continual learning algorithms are evaluated on their ability to maintain "intra-task" accuracy. If a model can still identify Task 1 classes when prompted, we say it hasn't forgotten. But in a real-world Class-Incremental Learning (CIL) scenario, the model isn't told which task an input belongs to. It must choose from all classes it has ever seen.
As Yiduo Guo and Dongyan Zhao argue in Dealing with Cross-Task Class Discrimination in Online Continual Learning, models often fail here because they never observe the decision boundaries between classes learned at different times. If Task A contains "cars" and Task B contains "trucks," and the model never sees them simultaneously, it has no reason to optimize the boundary between them. The model learns to separate cars from non-cars and trucks from non-trucks, but the "car" and "truck" feature spaces often overlap, leading to high-confidence misclassifications.
The Mathematical Gap: Unoptimized Off-Diagonals
To understand why this happens, we can look at the loss matrix of a continual learner. In Task Confusion and Catastrophic Forgetting in Class-Incremental Learning, Milad Khademi Nori and Il-Min Kim describe how standard training only optimizes the "diagonal blocks" of the total class-discrimination matrix.
When training on Task $N$, the model optimizes the boundaries between classes within Task $N$. The "off-diagonal" blocks—which represent the comparison between a class in Task $N$ and a class in Task $N-1$—remain unoptimized. Without simultaneous access to data from both tasks, the model cannot minimize the error associated with distinguishing between these disjoint sets. This is why weight regularization techniques like EWC or SI, which focus on keeping weights near their old values, often fail in CIL; they preserve the features but do not establish the relationships between tasks.
Strategies for Establishing Boundaries
Research has converged on several strategies to bridge this gap:
- Generative Replay: Instead of storing old data (which has privacy and storage costs), models can learn to generate it. Michiel van der Ven introduced Replay-through-Feedback (RtF) in Generative replay with feedback connections as a general strategy for continual learning, which integrates a generator directly into the classifier. By re-synthesizing past data, the model can "see" old and new classes together, allowing it to draw those missing decision boundaries.
- Task Prediction as OOD Detection: If a model can accurately predict which task an input belongs to, it can use task-specific parameters. Gyuhak Kim and Bing Liu proved in Learnability and Algorithm for Continual Learning that CIL is learnable if we treat task identification as an Out-of-Distribution (OOD) detection problem. By identifying an input as "not belonging to the current task," the model can avoid the confusion that plagues joint classification.
- Virtual Outliers and Contrastive Learning: Some methods attempt to sharpen boundaries without real data. In OVOR: OnePrompt with Virtual Outlier Regularization, Wei-Cheng Huang and colleagues use synthesized "virtual outliers" to define the space where old classes might exist, forcing the model to keep the new class representations compact and separable. Similarly, Haizhou Shi proposed C4IL, which uses label-guided contrastive learning to ensure that feature representations are inherently discriminative across phases.
Conclusion
The shift from "preventing forgetting" to "ensuring separation" marks a maturing of the field. As Alexander Gepperth and Timothée Lesort demonstrate in their work on Conditional Replay, the goal is no longer just stability, but the creation of a shared, global decision space where new knowledge is integrated, not just appended. Understanding ICS explains why simple regularization often fails and points toward a future where models learn the boundaries between tasks as explicitly as they learn the tasks themselves.