I2CANSAY:Inter-Class Analogical Augmentation and Intra-Class Significance Analysis for Non-Exemplar Online Task-Free Continual Learning

Songlin Dong, Yingjie Chen, Yuhang He, Yuhan Jin, Alex C. Kot, Yihong Gong

Abstract

Online Task-Free Continual Learning (OTFCL) is a challenging paradigm where models must learn from a seamless stream of data without explicit task boundaries or the ability to store past samples in memory buffers. We introduce I²CANSAY, a non-exemplar framework that eliminates the need for buffers by leveraging two novel components: Inter-Class Analogical Augmentation (ICAN) and Intra-Class Significance Analysis (ISAY). By synthesizing "pseudo-features" through analogical reasoning and correcting classifier bias via feature significance, I²CANSAY achieves state-of-the-art performance on image classification benchmarks without ever re-processing old data.

Introduction

Standard Continual Learning (CL) typically relies on "tasks" and "memory buffers" to combat catastrophic forgetting—the tendency of neural networks to overwrite old knowledge when learning new information. However, real-world data streams are rarely neatly partitioned, and storing user data in buffers often violates privacy regulations and hinders computational efficiency.

I²CANSAY addresses these limitations by focusing on a "non-exemplar" approach. Instead of storing images, it stores class prototypes (average feature vectors) and uses the variance of incoming data to "hallucinate" the distribution of past classes. This allows the model to maintain high accuracy in a one-shot learning environment where data is seen only once.

Methodology

The I²CANSAY framework consists of two primary modules designed to work with a pre-trained, frozen backbone:

1. Inter-Class Analogical Augmentation (ICAN)

ICAN operates on the premise that different object categories often share similar distribution patterns in the feature space, such as variations in pose, lighting, or background. When a new class is encountered, the model calculates its feature variance and applies this "variation" to the stored prototypes of old classes. This process generates high-quality pseudo-features that represent what the old classes might look like under the current data's conditions.

Conceptual overview of shared distributions and analogical reasoning between different species.

2. Intra-Class Significance Analysis (ISAY)

Learning from a single pass of data makes it difficult to distinguish between essential class attributes and noise. ISAY analyzes which dimensions of a feature vector are stable across samples of a class. Dimensions with low variance are deemed "significant" (e.g., the shape of a bear), while high-variance dimensions (e.g., color in varying light) are suppressed. This significance vector is used to replace the classifier's bias, effectively guiding the model to focus on robust identifying features.

The I²CANSAY architecture, highlighting the ICAN and ISAY modules and the generation of pseudo-features.

Key Results and Insights

The framework was evaluated on standard benchmarks including CIFAR-100, CoRe50, and the fine-grained CUB-200 dataset.

  • Superior Accuracy: On CIFAR-100 using a ResNet50 backbone, I²CANSAY achieved 57.3% accuracy, outperforming the previous state-of-the-art ensemble method (which required a 6,000-image buffer) by over 18% in realistic Gaussian distribution settings.
  • Vision Transformer (ViT) Synergy: When paired with a ViT backbone, the performance gap widened, with I²CANSAY leading other methods by up to 11.7%.
  • Data Efficiency: The model demonstrated remarkable stability even when training data was reduced to 10-30% of the original set, proving its robustness in data-scarce environments.

Conclusion

I²CANSAY demonstrates that memory buffers are not a prerequisite for effective continual learning. By treating variance as a signal rather than noise and using inter-class analogies to synthesize training data, the framework provides a privacy-preserving and efficient path forward for online learning. Future research will explore the applicability of these analogical augmentations in non-visual domains such as natural language processing and sensor data analysis.

Summary updated

BibTeX
@article{dong2025i,
  author = {Songlin Dong and Yingjie Chen and Yuhang He and Yuhan Jin and Alex C. Kot and Yihong Gong},
  title = {{I2CANSAY:Inter-Class Analogical Augmentation and Intra-Class Significance Analysis for Non-Exemplar Online Task-Free Continual Learning}},
  journal = {IEEE Trans. Multimedia},
  year = {2025},
  doi = {10.1109/tmm.2025.3535384},
}
Figure