TL;DR
This paper introduces a novel method for "online continual learning" that allows models to learn new tasks sequentially without forgetting old ones, even when only seeing each data point once. Their approach uses a "candidates voting" mechanism and stores feature embeddings instead of raw data, making it memory-efficient and privacy-preserving, outperforming existing methods on several benchmarks.
What Problem Does This Paper Address?
Continual learning aims to train models on a sequence of tasks without retraining from scratch. A particularly challenging scenario is "online class-incremental learning," where the model must classify instances from all previously learned classes, and each new data point can only be used once for training. This setup mimics real-world scenarios where data arrives sequentially and resources are limited. The main hurdles are:
- Catastrophic Forgetting: Models tend to forget previously learned knowledge when exposed to new data.
- Weight Bias: In class-incremental learning with a single classifier, the model's output logits become biased towards newer tasks, further exacerbating forgetting.
- Memory and Privacy Constraints: Many existing methods rely on storing original data samples (exemplars) from previous tasks for knowledge replay, which is often infeasible due to limited memory or privacy concerns.
- Limited Research in Online Setting: The online class-incremental setting is less explored compared to its offline counterpart.
Methods
The proposed method, "Online Continual Learning via Candidates Voting," addresses these challenges through two main components:
- Candidates Selection: Instead of relying solely on the highest output logit from a single-head classifier (which is prone to bias), the method identifies the highest logit for each learned task as a "candidate." This is achieved by using stored feature embeddings and task indices to generate binary masks that isolate the output logits for each task.
- Prior Incorporation: A Probabilistic Neural Network (PNN) is used to estimate the probability that an input belongs to each previously learned task. These probabilities serve as weights, which are then used to combine the selected candidates into a final prediction. This weighted voting mechanism helps to balance predictions across old and new tasks.
Key novelties include:
- Feature Embeddings as Exemplars: Storing feature embeddings instead of raw data significantly reduces memory footprint and enhances privacy.
- Online Sampler: A dynamic mean update criterion is used to select exemplars from the data stream, ensuring they are representative of class means without requiring prior knowledge of the total data size.
- Exemplar Augmentation (EA): Random perturbation is applied to feature embeddings in the feature space to augment the exemplar set, improving performance without increasing storage.
Datasets
The paper evaluates its method on the following datasets:
- CIFAR-10: Split into 5 tasks, each containing 2 disjoint classes.
- CIFAR-100: Split into 20 tasks with non-overlapping classes.
- CORE-50: A benchmark for continual object recognition, split into 9 tasks with a total of 50 classes.
Key Insights and Ideas
- Decoupling Prediction from Raw Logits: The core idea of using "candidates voting" effectively mitigates the weight bias problem by not solely relying on the single highest logit from the classifier, which is heavily influenced by recent tasks.
- Feature Embeddings are Sufficient: Storing feature embeddings, rather than raw images, is a practical and privacy-preserving way to retain knowledge for replay, significantly reducing memory requirements.
- Online Learning Can Compete with Offline: The paper demonstrates that their online method can achieve performance comparable to state-of-the-art offline methods, which is a significant achievement given the stricter constraints of online learning.
- Exemplar Augmentation is Crucial for Limited Storage: When memory is scarce, augmenting feature embeddings in the feature space provides a substantial performance boost without requiring additional storage for raw data.
Key Results
- Superior Performance: The proposed method consistently outperforms existing online continual learning approaches on Split CIFAR-10, Split CIFAR-100, and CORE-50, especially with smaller exemplar set sizes.
- Competitive with Offline Methods: On CIFAR-100, their online method achieves comparable performance to several state-of-the-art offline methods.
- Significant Memory Reduction: By storing feature embeddings, the method requires substantially less storage compared to methods that store raw data exemplars. For example, storage is O(D × C × Q) for their method versus O(S² × C × Q) for raw data storage, where D is feature dimension, C is number of classes, Q is exemplars per class, and S is image size.
- Weight Bias Mitigation: Analysis of weight vector norms shows that their method effectively reduces the weight bias problem, a key factor in catastrophic forgetting.
Limitations and Future Work
- Limitations:
- The paper acknowledges that while their method is effective, the performance in the online setting is generally lower than in the offline setting, as expected due to the data usage constraints.
- The method still relies on a fixed backbone network, which might limit its adaptability to drastically different future tasks.
- Future Work:
- The authors plan to focus on unsupervised continual learning, which is considered more realistic and could leverage pseudo-labeling techniques.
- Further exploration of exemplar augmentation strategies is suggested.
Summary updated