Introduction
In the field of Continual Learning (CL), the primary objective is typically to prevent "catastrophic forgetting"—the tendency of neural networks to lose previously acquired knowledge when learning new tasks. However, this objective becomes problematic when the incoming data stream contains noisy or incorrectly labeled samples. In such scenarios, standard Experience Replay (ER) methods inadvertently preserve and reinforce errors, leading to a significant degradation in model performance.
This paper introduces a paradigm shift by treating forgetting not as a failure, but as a strategic filter. The authors propose Alternate Experience Replay (AER), a framework that leverages the different rates at which models forget clean versus noisy data to effectively purify the memory buffer.
The Challenge: The Vanishing Gap in Online Learning
Existing methods for handling noisy labels often rely on the "small-loss" hypothesis: the observation that models learn clean patterns before they begin to memorize noise. In offline settings, one can simply filter out high-loss samples. However, in the "online" or streaming context of Continual Learning, the model adapts to new data so rapidly that the loss gap between clean and noisy samples quickly vanishes. Once the model has memorized a noisy label, it becomes indistinguishable from a clean one based on loss alone, rendering standard purification strategies ineffective.
Methodology
1. Alternate Experience Replay (AER)
To address the vanishing gap, the authors propose an alternating training regime. AER cycles between two distinct modes:
- Buffer Learning: The model undergoes standard training using a combination of the current data stream and the samples stored in the memory buffer.
- Buffer Forgetting: The model temporarily ceases training on the buffer and focuses exclusively on the current data stream.
Because neural networks are more prone to forgetting incorrectly labeled data when those labels are not actively reinforced, the loss values for noisy samples in the buffer spike during the forgetting phase. This "resets" the loss distribution, allowing the system to once again distinguish between reliable signal and noise.
2. Asymmetric Balanced Sampling (ABS)
Once noise is identified, the system must decide which samples to retain in the limited memory buffer. The authors introduce Asymmetric Balanced Sampling (ABS), which applies different selection criteria based on the age of the task:
- Current Task (Purity): For the most recent data, the priority is to ensure the buffer is not poisoned. The system uses a strict small-loss criterion to select only the cleanest samples.
- Past Tasks (Diversity): For older tasks, the priority shifts to maintaining a representative sample of the decision boundary. The system retains "complex" (higher-loss) clean samples, as these are more informative for preventing future forgetting of old tasks.
3. Buffer Consolidation
At the conclusion of each task, the framework performs a refinement step. By fitting a Gaussian Mixture Model (GMM) to the loss distribution of the buffer, the model separates "pure" samples from "uncertain" ones. It then employs a MixMatch-based semi-supervised learning approach to further harden the model against any residual noise.
Experimental Evaluation
The researchers evaluated AER/ABS across several benchmarks, including Seq. CIFAR-10, Seq. CIFAR-100, and the Seq. NTU RGB+D action recognition dataset. They also tested the framework on Seq. Food-101N, a real-world dataset containing naturally occurring instance-level noise from web scraping.
Key Results
- Performance: AER with ABS achieved a 4.71% average accuracy improvement over existing state-of-the-art loss-based purification strategies.
- Computational Efficiency: The method is remarkably faster than its competitors. On CIFAR-10, AER required only 1.5 hours of training, whereas the adapted SPR baseline would have required over 119 hours to achieve comparable results.
- Buffer Quality: Analysis of the memory buffers showed that the asymmetric approach successfully balanced high purity (low noise) with high diversity (class representation), outperforming standard uniform sampling methods.
Conclusion
By reframing forgetting as a tool for data purification, the authors provide a robust solution for Continual Learning in noisy environments. The AER framework effectively "stretches" the loss distribution to expose mislabeled data, while the ABS strategy ensures that the memory buffer remains both clean and informative. This "plug-and-play" approach offers a scalable and efficient path forward for deploying AI systems in real-world scenarios where perfectly labeled data is a luxury.
Summary updated