Few-Shot Class-Incremental Learning via Feature Space Composition

Hanbin Zhao, Yongjian Fu, Xuewei Li, Songyuan Li, Bourahla Omar, Xi Li

Abstract

Few-Shot Class-Incremental Learning (FSCIL) requires models to learn new classes from limited data without forgetting previously acquired knowledge. This paper introduces the Multi-Grained Slow vs. Fast (MgSvF) framework, which addresses the stability-plasticity dilemma by disentangling features in the frequency domain and composing dual feature spaces. By balancing "slow" preservation and "fast" adaptation, the method achieves state-of-the-art performance, outperforming previous methods by over 10% on challenging benchmarks.

The SvF Dilemma in FSCIL

In machine learning, models often struggle with a fundamental trade-off when learning incrementally. To retain old knowledge, a model must change its parameters slowly, which can lead to underfitting on new data. Conversely, to learn from just a few examples, the model must update quickly, which often triggers "catastrophic forgetting."

Conceptual comparison of (a) Slow forgetting, (b) Fast adaptation, and (c) the proposed SvF approach which balances both.

Most existing methods attempt to find a compromise within a single feature space. However, because feature dimensions are often mutually correlated, it is difficult to isolate which components should remain stable and which should adapt. The MgSvF framework addresses this by operating at two distinct levels: intra-space and inter-space.

The MgSvF Framework

1. Intra-space SvF: The Frequency Domain

The researchers discovered that different components of a feature vector exhibit varying levels of stability. By applying a Discrete Cosine Transform (DCT), they move the data into an orthogonal frequency domain.

Empirical analysis shows that low-frequency components are more stable and better at preserving old knowledge, while high-frequency components are more prone to forgetting but offer more flexibility for adaptation.

Average forgetting percentage across different frequency components, showing that higher frequencies are more susceptible to forgetting over 10 incremental sessions.

Based on this insight, the model implements frequency-aware regularization. During incremental steps, a heavy penalty is applied to changes in low-frequency components (keeping them "slow"), while high-frequency components are allowed more freedom to adapt ("fast").

2. Inter-space SvF and Composition

Beyond frequency-level adjustments, the framework maintains two distinct feature spaces:

  • Slow-Updated Space: Trained with a very low learning rate to act as a stable anchor for old knowledge.
  • Fast-Updated Space: Trained with a higher learning rate to aggressively capture the nuances of new, few-shot classes.

Finally, the model merges these spaces using a composition operation (such as concatenation or PCA-based reduction) to create a "Composite Feature Space." Classification is then performed using a Nearest Class Mean (NCM) classifier, comparing samples against class prototypes.

Experimental Results

The MgSvF framework was evaluated on three standard benchmarks: CIFAR100, CUB200-2011, and MiniImageNet. The results demonstrate significant improvements over previous state-of-the-art methods:

  • CUB200: The method achieved a final accuracy of 54.33% after 11 learning sessions, a 11.75% improvement over the previous best (SDC at 42.58%).
  • Stability: On CIFAR100 and MiniImageNet, the "forgetting curve" remained significantly flatter than baseline methods, indicating superior knowledge retention.
  • Efficiency: Despite the dual-space architecture, training time only increased by approximately 2%, making the approach highly practical.

Conclusion

The MgSvF framework demonstrates that disentangling features into "slow" and "fast" components—both within a single space via frequency analysis and across dual architectures—is a powerful strategy for FSCIL. By leveraging the complementary nature of stable and adaptable features, the model effectively mitigates catastrophic forgetting while enabling rapid learning from limited data.

Summary updated

Abstract

As a challenging problem, few-shot class-incremental learning (FSCIL) continually learns a sequence of tasks, confronting the dilemma between slow forgetting of old knowledge and fast adaptation to new knowledge. In this paper, we concentrate on this “slow versus fast” (SvF) dilemma to determine which knowledge components to be updated in a slow fashion or a fast fashion, and thereby balance old-knowledge preservation and new-knowledge adaptation. We propose a multi-grained SvF learning strategy to cope with the SvF dilemma from two different grains: intra-space (within the same feature space) and inter-space (between two different feature spaces). The proposed strategy designs a novel frequency-aware regularization to boost the intra-space SvF capability, and meanwhile develops a new feature space composition operation to enhance the inter-space SvF learning performance. With the multi-grained SvF learning strategy, our method outperforms the state-of-the-art approaches by a large margin.
BibTeX
@article{zhao2024few,
  author = {Hanbin Zhao and Yongjian Fu and Xuewei Li and Songyuan Li and Bourahla Omar and Xi Li},
  title = {{Few-Shot Class-Incremental Learning via Feature Space Composition}},
  journal = {IEEE Trans. Pattern Anal. Mach. Intell.},
  year = {2024},
  doi = {10.1109/tpami.2021.3133897},
}
Figure