Few-Shot Lifelong Learning

Pratik Mazumder, Pravendra Singh, Piyush Rai

Abstract

Few-Shot Class-Incremental Learning (FSCIL) presents a dual challenge: models must learn new categories from extremely limited data (Few-Shot Learning) while simultaneously preventing the erasure of previously acquired knowledge (Catastrophic Forgetting). We introduce Few-Shot Lifelong Learning (FSLL), a surgical approach that identifies and updates only "unimportant" parameters to accommodate new information. By preserving foundational weights and actively decoupling new class representations from old ones through cosine similarity and self-supervision, FSLL significantly outperforms existing benchmarks, achieving a nearly 20% improvement on fine-grained classification tasks.

Introduction

Standard deep learning models excel in static environments but falter when deployed in dynamic, real-world scenarios. Two primary issues arise: overfitting on scarcity, where models memorize sparse examples of new classes instead of generalizing, and catastrophic forgetting, where sequential learning of new tasks overwrites the weights essential for previous ones. FSCIL combines these difficulties, requiring a model to expand its vocabulary of classes with minimal data without degrading its base performance.

Methodology

The FSLL framework employs a three-pronged strategy to address the stability-plasticity dilemma in incremental learning.

Surgical Parameter Selection

Rather than retraining the entire network, FSLL identifies "important" and "unimportant" weights based on their absolute magnitude after initial base training. During incremental sessions, the "important" weights—which represent foundational features—are frozen to ensure knowledge retention. Only the "unimportant" weights, acting as a "scratchpad," are updated to learn new classes.

Overview of the FSLL framework showing parameter selection and feature space separation across incremental sessions.

Active Feature Decoupling

To maintain clean decision boundaries, FSLL implements a loss function that minimizes the cosine similarity between the prototypes (mean feature vectors) of new classes and those of previously learned classes. This "active decoupling" pushes new categories into unoccupied regions of the feature space, preventing confusion between old and new tasks.

Robust Feature Foundation

The model utilizes self-supervised auxiliary tasks, specifically rotation prediction, during the initial training phase. This forces the network to learn high-quality, robust features that are more adaptable to future few-shot tasks compared to standard supervised learning or other self-supervised methods like SimCLR.

Experimental Evaluation

The method was rigorously evaluated on three standard benchmarks: CIFAR-100 (general objects), miniImageNet (diverse categories), and CUB-200 (fine-grained bird species). The CUB-200 dataset is particularly demanding due to the high visual similarity between classes.

Key Results

FSLL established a new state-of-the-art across all datasets. The most significant gains were observed in fine-grained classification:

  • CUB-200: FSLL outperformed the previous leading method (TOPIC) by an absolute margin of 19.27% in final accuracy.
  • miniImageNet: Achieved an improvement of approximately 15.07% over existing methods.
  • CIFAR-100: Demonstrated a 9.09% performance boost.
Performance comparison on the CUB-200 dataset across 11 sessions, highlighting relative improvements over baselines.

Ablation studies confirmed that updating only 10% of the parameters provided the optimal balance between learning capacity and forgetting prevention.

Conclusion and Future Work

FSLL demonstrates that a targeted, surgical approach to weight updates is highly effective for lifelong learning. By treating low-magnitude weights as adaptable resources and actively separating feature representations, the model maintains high performance across long sequences of tasks.

Future research may explore more sophisticated parameter selection heuristics beyond absolute magnitude, such as gradient-based sensitivity. Additionally, addressing potential feature space saturation as the number of classes grows remains a critical challenge for scaling lifelong learning systems.

Summary updated

Abstract

Many real-world classification problems often have classes with very few labeled training samples. Moreover, all possible classes may not be initially available for training, and may be given incrementally. Deep learning models need to deal with this two-fold problem in order to perform well in real-life situations. In this paper, we propose a novel Few-Shot Lifelong Learning (FSLL) method that enables deep learning models to perform lifelong/continual learning on few-shot data. Our method selects very few parameters from the model for training every new set of classes instead of training the full model. This helps in preventing overfitting. We choose the few parameters from the model in such a way that only the currently unimportant parameters get selected. By keeping the important parameters in the model intact, our approach minimizes catastrophic forgetting. Furthermore, we minimize the cosine similarity between the new and the old class prototypes in order to maximize their separation, thereby improving the classification performance. We also show that integrating our method with self-supervision improves the model performance significantly. We experimentally show that our method significantly outperforms existing methods on the miniImageNet, CIFAR-100, and CUB-200 datasets. Specifically, we outperform the state-of-the-art method by an absolute margin of 19.27% for the CUB dataset.
BibTeX
@inproceedings{mazumder2021few,
  author = {Pratik Mazumder and Pravendra Singh and Piyush Rai},
  title = {{Few-Shot Lifelong Learning}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2021},
  doi = {10.1609/aaai.v35i3.16334},
}
Figure