DYSON: Dynamic Feature Space Self-Organization for Online Task-Free Class Incremental Learning

Yuhang He, Yingjie Chen, Yuhan Jin, Songlin Dong, Xing Wei, Yihong Gong

Introduction

Online Task-Free Class Incremental Learning (OTFCIL) is a challenging frontier in machine learning where models must learn from a continuous data stream without explicit task boundaries. The primary obstacle is catastrophic forgetting: as new classes are introduced, the model's feature space typically shifts, causing it to lose previously acquired knowledge. Traditional solutions often rely on large memory buffers to "replay" old samples, which is inefficient and often impractical due to storage or privacy constraints.

The DYSON Paradigm: Compute-and-Align

DYSON (DYnamic feature space Self-OrganizatioN) introduces a novel "compute-and-align" paradigm. Instead of allowing the feature space to drift unpredictably as new data arrives, DYSON pre-computes an optimal geometric structure based on Neural Collapse theory and forces the network to align its features to this stable target.

Comparison of (a) traditional data-driven learning vs. (b) the proposed compute-and-align paradigm.

The framework consists of three core components:

  1. Dynamic Neural Collapse (DNC): Leveraging the phenomenon where deep networks naturally map classes to symmetric geometric shapes (Simplex Equiangular Tight Frames), DNC mathematically calculates where class prototypes should be. It expands this geometry as new classes arrive while providing a mathematical guarantee on the stability of old prototypes.
  2. Dynamic Feature-Geometry Alignment (DFGA): DYSON utilizes a frozen, pre-trained backbone (such as ResNet or ViT) and trains only a small projection layer to map these features onto the fixed, optimal coordinates defined by DNC.
  3. Prototype Placeholders: These reserve space in the geometry for future classes, ensuring long-term stability and preventing "jitter" when the model has only seen a few classes.

Experimental Results

DYSON was evaluated on several standard computer vision benchmarks, including CIFAR-10, CIFAR-100, CUB-200, and CoRe50. The model consistently outperformed state-of-the-art (SOTA) methods across various backbones and incremental steps.

Performance comparison of DYSON against SOTA methods on CIFAR-10 and CIFAR-100 across various backbones.

Key results include:

  • Significant Accuracy Gains: On CIFAR-100 using a ViT backbone, DYSON outperformed the previous best method (DSDM) by a massive 16.6% in accuracy in the 1-step setting.
  • Memory-Free Efficiency: DYSON achieves SOTA results with zero stored samples, outperforming many methods that rely on large memory buffers for data replay.
  • Backbone Agnostic: The method demonstrated robust performance whether using traditional ResNet50 or modern Vision Transformer (ViT) architectures.

Conclusion and Future Work

DYSON demonstrates that shifting from learning a feature space to computing one can effectively mitigate catastrophic forgetting in online learning scenarios. By pre-reserving slots in an optimal geometric layout, the model maintains stability without the need for raw data replay. Future research may explore adapting the backbone itself through parameter-efficient fine-tuning (PEFT) to further improve performance in domains far removed from the pre-training data.

Summary updated

Abstract

In this paper, we focus on a challenging Online Task-Free Class Incremental Learning (OTFCIL) problem. Dif-ferent from the existing methods that continuously learn the feature space from data streams, we propose a novel compute-and-align paradigm for the OTFCIL. It first com-putes an optimal geometry, i.e., the class prototype distri-bution, for classifying existing classes and updates it when new classes emerge, and then trains a DNN model by aligning its feature space to the optimal geometry. To this end, we develop a novel Dynamic Neural Collapse (DNC) algorithm to compute and update the optimal geometry. The DNC ex-pands the geometry when new classes emerge without loss of the geometry optimality and guarantees the drift distance of old class prototypes with an explicit upper bound. On this basis, we propose a novel DYnamic feature space Self-OrganizatioN (DYSON) method containing three ma-jor components, including 1) a feature extractor, 2) a Dy-namic Feature-Geometry Alignment (DFGA) module aligning the feature space to the optimal geometry computed by DNC and 3) a training-free class-incremental classifier de-rived from the DNC geometry. Experimental comparison results on four benchmark datasets, including CIFAR10, CI-FAR100, CUB200, and CoRe50, demonstrate the efficiency and superiority of the DYSON method. The source code is released at https://github.com/isCDX2IDYSON.
BibTeX
@inproceedings{he2024dyson,
  author = {Yuhang He and Yingjie Chen and Yuhan Jin and Songlin Dong and Xing Wei and Yihong Gong},
  title = {{DYSON: Dynamic Feature Space Self-Organization for Online Task-Free Class Incremental Learning}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2024},
  doi = {10.1109/cvpr52733.2024.02241},
}
Figure