Abstract
Few-Shot Class Incremental Learning (FSCIL) presents a dual challenge: preventing catastrophic forgetting of old classes while avoiding overfitting on new classes with limited samples. Conventional approaches often result in a crowded feature space that leaves little room for incremental expansion. We present FACL (Feature Augmentation driven Contrastive Learning), a novel framework that proactively manages feature space during the initial base training session. By generating synthetic proxy classes through feature blending, FACL reserves "strategic vacancies" in the representation map, facilitating the seamless integration of future categories. Our approach achieves state-of-the-art performance on CIFAR100, miniImageNet, and CUB200 benchmarks.
The Challenge of Incremental Learning
Standard deep learning models are notoriously rigid. When tasked with learning new information—such as a facial recognition system learning a new user from just five photos—they typically encounter two hurdles: Catastrophic Forgetting, where new weights overwrite old knowledge, and Overfitting, where the model memorizes the few available samples rather than generalizing.
Existing solutions often "freeze" the model after initial training to prevent forgetting. However, because these models are unaware of future classes during the base session, they pack the original classes tightly together. This lack of foresight leads to a "crowded" feature space where new classes inevitably overlap with old ones, causing significant classification errors as the model scales.
The FACL Framework
The FACL framework shifts the focus to the initial "base" training session, making it more strategic. Instead of just learning to distinguish current classes, the model learns to leave "gaps" in its knowledge map. The system is built on three main pillars:
- Feature Augmentation: Rather than simple image-level transforms (like rotation), FACL blends feature vectors from two different augmented versions of an image using a $\delta$ coefficient. This creates a "mixture" feature that preserves high-level semantic structures while appearing novel to the model.
- Proxy Class Generation: These blended features are assigned "proxy labels," forcing the model to treat them as entirely new categories. This effectively reserves placeholders in the feature space for classes it hasn't seen yet.
- Self-Supervised Contrastive Learning (SSCL): Utilizing a MoCo-based architecture, the model is trained to pull similar samples closer together and push dissimilar ones (including the proxy classes) further apart. This ensures that the boundaries between classes are wide and distinct.
At inference time, the model employs a Nearest-Mean Classifier (NCM), comparing a new image's features to the average "prototype" of each known class to make a prediction.
Experimental Results
FACL was validated on three standard computer vision benchmarks: CIFAR100, miniImageNet, and CUB200. The framework achieved state-of-the-art (SOTA) performance across all tested datasets:
- CIFAR100: Achieved a 4.77% average accuracy improvement over previous methods and a 2.87% boost in the final session.
- CUB200: Saw a 1.59% average accuracy improvement, with the final session performing 2.2% better than the previous best method.
Visual analysis via t-SNE plots confirms that FACL creates much tighter, more distinct clusters with wider "no-man's-land" zones between them compared to standard Cross-Entropy training. These vacancies are precisely where new, real classes are "slotted in" later without causing a collision with existing knowledge.
Discussion and Limitations
A recurring theme in this research is that the quality of the first training session determines the success of all subsequent incremental steps. While FACL shows a higher "Performance Drop" (PD) metric, this is a mathematical paradox: because FACL starts with such high accuracy in the base session, the relative drop when new classes are added appears larger, even though the final absolute accuracy remains higher than competitors.
Currently, the approach relies on freezing the backbone after the base session to prevent forgetting. Future work may explore ways to allow the backbone to evolve slightly, potentially reducing the performance drop while maintaining the benefits of the reserved feature space.
Summary updated