Abstract
Online Task-Free Continual Learning (OTFCL) is a challenging paradigm where models must learn from a seamless stream of data without explicit task boundaries or the ability to store past samples in memory buffers. We introduce I²CANSAY, a non-exemplar framework that eliminates the need for buffers by leveraging two novel components: Inter-Class Analogical Augmentation (ICAN) and Intra-Class Significance Analysis (ISAY). By synthesizing "pseudo-features" through analogical reasoning and correcting classifier bias via feature significance, I²CANSAY achieves state-of-the-art performance on image classification benchmarks without ever re-processing old data.
Introduction
Standard Continual Learning (CL) typically relies on "tasks" and "memory buffers" to combat catastrophic forgetting—the tendency of neural networks to overwrite old knowledge when learning new information. However, real-world data streams are rarely neatly partitioned, and storing user data in buffers often violates privacy regulations and hinders computational efficiency.
I²CANSAY addresses these limitations by focusing on a "non-exemplar" approach. Instead of storing images, it stores class prototypes (average feature vectors) and uses the variance of incoming data to "hallucinate" the distribution of past classes. This allows the model to maintain high accuracy in a one-shot learning environment where data is seen only once.
Methodology
The I²CANSAY framework consists of two primary modules designed to work with a pre-trained, frozen backbone:
1. Inter-Class Analogical Augmentation (ICAN)
ICAN operates on the premise that different object categories often share similar distribution patterns in the feature space, such as variations in pose, lighting, or background. When a new class is encountered, the model calculates its feature variance and applies this "variation" to the stored prototypes of old classes. This process generates high-quality pseudo-features that represent what the old classes might look like under the current data's conditions.
2. Intra-Class Significance Analysis (ISAY)
Learning from a single pass of data makes it difficult to distinguish between essential class attributes and noise. ISAY analyzes which dimensions of a feature vector are stable across samples of a class. Dimensions with low variance are deemed "significant" (e.g., the shape of a bear), while high-variance dimensions (e.g., color in varying light) are suppressed. This significance vector is used to replace the classifier's bias, effectively guiding the model to focus on robust identifying features.
Key Results and Insights
The framework was evaluated on standard benchmarks including CIFAR-100, CoRe50, and the fine-grained CUB-200 dataset.
- Superior Accuracy: On CIFAR-100 using a ResNet50 backbone, I²CANSAY achieved 57.3% accuracy, outperforming the previous state-of-the-art ensemble method (which required a 6,000-image buffer) by over 18% in realistic Gaussian distribution settings.
- Vision Transformer (ViT) Synergy: When paired with a ViT backbone, the performance gap widened, with I²CANSAY leading other methods by up to 11.7%.
- Data Efficiency: The model demonstrated remarkable stability even when training data was reduced to 10-30% of the original set, proving its robustness in data-scarce environments.
Conclusion
I²CANSAY demonstrates that memory buffers are not a prerequisite for effective continual learning. By treating variance as a signal rather than noise and using inter-class analogies to synthesize training data, the framework provides a privacy-preserving and efficient path forward for online learning. Future research will explore the applicability of these analogical augmentations in non-visual domains such as natural language processing and sensor data analysis.
Summary updated