Abstract
Class Incremental Learning (CIL) aims to enable deep learning models to learn new classes sequentially without forgetting previously acquired knowledge. While most research focuses on mitigating "catastrophic forgetting" during the incremental update phase, this paper proposes that the robustness of the initial base model is paramount. By employing Multitask Learning (MTL) during the base training stage, the authors develop a feature extractor with superior generalizability. This approach, combined with a strategic two-step fine-tuning process, significantly improves incremental learning performance on audio classification tasks, outperforming existing state-of-the-art methods.
Introduction
Deep learning models are traditionally static, trained on fixed datasets. When these models encounter new data in a Class Incremental Learning (CIL) setting, they often suffer from "catastrophic forgetting," where new information overwrites previously learned features. Most existing solutions attempt to "patch" the model during incremental steps using knowledge distillation or exemplar replay. However, this research shifts the focus to the base model training stage, arguing that a more generalizable initial foundation is the key to resilient incremental learning.
Methodology: Multitask Base Training
The core innovation lies in using Multitask Learning (MTL) to train the initial base model. Instead of a single monolithic classifier, the authors decompose the base classes into multiple overlapping subsets, each treated as a distinct task.
As shown in the figure above, all tasks share a single feature extractor (backbone) but utilize separate classification heads. This architecture forces the backbone to learn features that are useful across diverse "views" of the data, resulting in a more transferable and robust embedding.
When new classes arrive during the incremental stage, the authors employ a Two-Step Fine-Tuning strategy:
- Backbone Freezing: The feature extractor is frozen while only the new classification neurons are trained with a high learning rate.
- Global Fine-Tuning: Once the new neurons converge, the entire network is unfrozen and fine-tuned with a very small learning rate to integrate new knowledge while preserving the old.
Experimental Results
The method was evaluated on two major audio datasets: UrbanSound8K and Google Speech Commands (GSC). These datasets represent real-world challenges for edge devices that must learn new environmental sounds or voice commands over time.
Performance Gains
The multitask approach yielded significant improvements in average incremental learning accuracy:
- Google Speech Commands: Up to 5.5% improvement over state-of-the-art baselines.
- UrbanSound8K: Approximately 3.2% improvement.
The confusion matrices illustrate the model's stability. The green dashed box represents the base classes, while the red dashed box tracks the incremental classes. Over time, the model successfully populates the diagonal for new classes without significantly degrading the performance on the initial set.
Key Insights
- The Diversity Hypothesis: Forcing the model to solve multiple sub-problems simultaneously prevents it from learning "shortcuts" or features specific to only one class set. This leads to a more general understanding of the data manifold.
- Synergy with Exemplars: The MTL-trained base model shows even greater performance gains when combined with traditional "exemplar replay" techniques, suggesting that a better foundation enhances the effectiveness of existing CIL strategies.
- Task Distinctness: The authors found that the most significant gains come from tasks that are distinct from one another. Highly overlapping tasks provide diminishing returns, emphasizing the need for diverse "perspectives" during training.
Conclusion and Future Work
This research demonstrates that the quality of the initial feature representation is a critical bottleneck for Class Incremental Learning. By leveraging Multitask Learning at the start, models can develop the flexibility needed to accommodate future growth. Future research will explore automated methods for grouping classes based on semantic meaning and extending this framework to large-scale computer vision benchmarks.
Summary updated