SATHUR: Self Augmenting Task Hallucinal Unified Representation for Generalized Class Incremental Learning

Sathursan Kanagarajah, Thanuja D. Ambegoda, Ranga RodrigoWorkshops @ ICCV 2023

Introduction: The Challenge of Generalized Class Incremental Learning

Standard Class Incremental Learning (CIL) often operates under idealized assumptions: fixed task sizes, no overlap between tasks, and perfectly balanced data. However, real-world scenarios—referred to as Generalized CIL (GCIL)—are far more complex. In GCIL, data is often imbalanced, and task boundaries are fluid, mirroring how humans naturally acquire knowledge.

Illustration of task boundaries and feature mapping in Generalized Class Incremental Learning.

The primary technical hurdle in GCIL is feature drift. As a neural network learns new tasks, its internal weights shift, effectively "invalidating" the feature representations of previously learned classes. SATHUR addresses this by focusing on re-aligning these representations rather than merely trying to freeze the network weights.

Methodology: The SATHUR Framework

SATHUR (Self Augmenting Task Hallucinal Unified Representation) introduces a three-pronged approach to maintain knowledge across shifting feature spaces by combining dynamic classification with synthetic feature generation.

The SATHUR workflow: (a) incremental training of the feature extractor and (b) GWR training using hallucinated features.

1. Grow When Required (GWR) Network

Instead of a standard static softmax classifier, SATHUR employs a GWR network—a dynamic type of Self-Organizing Map. GWR can add or remove nodes and edges based on the input data, making it inherently robust to the class imbalances and varying sample sizes typical of GCIL. Because GWR only updates nodes in the immediate neighborhood of an input, it naturally resists global catastrophic interference.

2. The Hallucinator

The "hallucinator" is the core innovation of SATHUR. When the CNN backbone is updated for a new task, the hallucinator generates synthetic feature vectors. These "hallucinated" samples bridge the gap between old GWR nodes and the updated feature extractor, ensuring that previous knowledge remains accessible in the new feature space. This process essentially "translates" old memories into the new "language" of the updated CNN.

3. Mixup Augmentation

To combat data scarcity in underrepresented classes, the framework utilizes Mixup augmentation. By performing linear interpolation between samples, SATHUR creates a more balanced training set for the feature extractor, improving its overall robustness.

Experimental Evaluation

The authors evaluated SATHUR on two rigorous benchmarks designed to simulate real-world learning challenges:

  • CIFAR-100 (Longtail): Structured into 20 incremental tasks with a long-tail distribution to simulate real-world imbalance.
  • CORe50: A continuous object recognition dataset consisting of video sequences, structured into 10 incremental tasks.

Key Results

SATHUR demonstrated superior performance over state-of-the-art methods like ReMix, Mnemonics, and MRDC:

  • CIFAR-100: Achieved 39.42% accuracy, a 3.70% improvement over the previous best (MRDC).
  • CORe50: Reached 69.54% accuracy, outperforming existing methods by 2.88% and approaching the "Full" training upper bound (training on all data at once).

Conclusion and Future Directions

SATHUR proves that re-aligning representations via hallucination is a powerful alternative to weight-freezing techniques in incremental learning. While highly effective, future research could focus on optimizing the memory buffer of exemplars and automating hyperparameter tuning for the GWR's activity thresholds. Its modular nature suggests it could serve as a valuable "plugin" for various replay-based architectures across different backbones.

Summary updated

BibTeX
@inproceedings{kanagarajah2023sathur,
  author = {Sathursan Kanagarajah and Thanuja D. Ambegoda and Ranga Rodrigo},
  title = {{SATHUR: Self Augmenting Task Hallucinal Unified Representation for Generalized Class Incremental Learning}},
  booktitle = {Workshops @ ICCV},
  year = {2023},
}
Figure