Introduction
Deep learning models typically operate under the assumption of a static dataset. However, real-world applications often require models to learn from a continuous stream of data where new classes appear over time without full access to previous training samples. This paradigm, known as Class-Incremental Learning (CIL), is plagued by catastrophic forgetting: as a model adapts to new information, it tends to overwrite the weights necessary for recognizing older classes, leading to a sharp decline in performance on previous tasks.
To address this, the DILF-EN framework introduces a novel approach that leverages image orientations as a stable "anchor" to preserve knowledge across learning phases.
The DILF-EN Framework
The framework consists of two synergistic components: Dual-Incremental Learning (DILF) and Data-Ensemble (EN).
Dual-Incremental Learning (DILF)
Standard CIL models focus solely on predicting image classes. DILF augments this by adding a secondary objective: orientation classification. During training, images are rotated (e.g., 0°, 90°, 180°, 270°) and assigned two labels: their semantic class and their rotation class.
While the set of image classes grows with each phase, the orientation classes remain constant. This consistency forces the backbone feature extractor to maintain stable, generalizable representations that are less susceptible to the weight drift caused by learning new, volatile image classes.
Data-Ensemble (EN)
At inference time, the model utilizes a form of Test-Time Augmentation (TTA) optimized for the DILF regime. Instead of a single pass, the model processes multiple orientations of a test image, generates class probabilities for each, and averages them to produce a final prediction.
Key Insights
- Orientation-Dependent Forgetting: A core discovery of this research is that a model may "forget" a class in its original orientation while still correctly identifying it when rotated. This suggests that class knowledge is often inconsistent rather than entirely lost.
- The "Anchor" Effect: Orientation serves as a fixed target. By training on a task that never changes, the model anchors its weights, preventing them from drifting too far during incremental updates.
- Mandatory Synergy: The authors emphasize that Data-Ensemble (EN) is ineffective without DILF training. Standard models, not trained on rotated inputs, often see a performance drop when subjected to ensemble testing across orientations.
Experimental Results
The framework was evaluated on standard benchmarks, including CIFAR-100 and ImageNet-100, across various phase counts ($P=5, 10, 25$). DILF-EN was integrated into several state-of-the-art (SOTA) baselines, including AANets, PODNet, and LUCIR.
As shown in the results, DILF-EN consistently improves performance across all settings. For instance, on CIFAR-100, it boosted the performance of AANets by absolute margins ranging from 3.30% to 4.28%. Visual analysis via Grad-CAM further revealed that DILF-EN helps the model maintain focus on discriminative object features rather than drifting toward background noise after multiple learning phases.
Limitations and Future Work
While effective, the framework introduces computational overhead. Training requires larger effective batch sizes to accommodate rotated images, and inference time increases due to the multi-pass ensemble process. Future research may explore other self-supervised tasks beyond rotation to provide even more robust anchors against catastrophic forgetting.
Summary updated