Abstract
Class-Incremental Learning (Class-IL) aims to learn new concepts sequentially without forgetting previous ones, a task significantly more difficult than Task-Incremental Learning (Task-IL) where task identifiers are provided at test time. This paper introduces a Dynamic Inference Network that utilizes a multi-centroid task descriptor to accurately predict task IDs during inference. By employing a gate network trained with bipartite matching and knowledge distillation, the proposed method bridges the performance gap between Class-IL and Task-IL, achieving state-of-the-art accuracy on large-scale benchmarks including ImageNet-1000.
Introduction
Incremental learning (IL) is essential for developing AI systems that can adapt to evolving data streams. While Task-IL provides the model with task context during testing, Class-IL requires the model to distinguish between all learned classes without such priors. The authors identify a significant "performance gap" where even state-of-the-art Class-IL models perform substantially better if provided with the task ID. The challenge in bridging this gap lies in the arbitrary nature of task groupings, which often lack semantic cohesion, making single-prototype task descriptors ineffective for capturing the high variance of unrelated classes within a single task.
Methodology
The proposed solution is a Dynamic Inference Network featuring a standalone gate network designed to predict task IDs and route inputs to the most relevant model branches.
Multi-Centroid Task Descriptor
Recognizing that classes within a task may be semantically diverse, the authors represent each task using multiple trainable centroids rather than a single point. This allows the descriptor to capture the disjoint clusters inherent in task-based data, effectively modeling the "disjointed" nature of arbitrary task groupings.
Bipartite Matching and Training
To ensure centroids remain representative and diverse, the Hungarian Algorithm is used during training to match class prototypes to specific centroids. This prevents centroid collapse and ensures full coverage of the task's semantic space. The training proceeds in two stages:
- Supernet Training: A multi-branch backbone (using DER as a baseline) is trained to learn features.
- Gate Training: The gate network is trained independently. To overcome the lack of old-task data, the authors use RandAugment for data synthesis and Knowledge Distillation to inherit semantic knowledge from the already-trained supernet.
Adaptive Inference
During inference, the gate network computes similarities between input features and task centroids. These similarities weight the outputs of the corresponding task branches. This mechanism not only improves accuracy but also allows for efficiency gains; by activating only the "Top-N" most likely branches, inference time can be reduced to just 10% of the baseline with negligible accuracy loss.
Experiments and Results
The method was validated on standard benchmarks including CIFAR-100, ImageNet-100, and the full-scale ImageNet-1000 dataset.
Key Results
The approach consistently outperformed existing baselines. On CIFAR100-B0S50 (50 steps), it achieved an average accuracy of 72.41%, a 3.40% improvement over the DER baseline. On the 1,000-class ImageNet challenge, it reached a top-1 average accuracy of 70.80%, surpassing competitive models like DyTox and FOSTER.
Ablation studies confirmed that the multi-centroid approach provides a ~1.5% boost over simple flat classification, validating the importance of the cluster-based descriptor.
Conclusion
By treating task identification as a dynamic routing problem supported by multi-centroid descriptors, this work effectively narrows the gap between Class-IL and Task-IL. While the gate network introduces a parameter overhead (approximately 11.2M parameters for ResNet-18), the gains in accuracy and inference efficiency make it a robust solution for large-scale incremental learning. Future research may explore merging similar branches to reduce the overall parameter footprint as the model grows.
Summary updated