In continual learning, a model encounters a sequential stream of data and must learn new tasks without forgetting old ones. The most common and effective remedy for this "catastrophic forgetting" is experience replay: storing a small fraction of past data in a memory buffer and periodically interleaving it with new data during training.
However, this elegant solution relies on a dangerous assumption: that the incoming data stream is perfectly clean. When the data stream contains noise—such as mislabeled samples, synthetic data, or ambiguous backgrounds—the memory buffer transforms from a stabilizing mechanism into an error amplifier.
Why Buffer Poisoning Causes Collapse
To understand why noise is uniquely destructive in continual learning, consider the feedback loop of rehearsal. In standard offline learning, a model might encounter a noisy label, compute an incorrect gradient, and eventually average it out over thousands of clean samples. But in a replay buffer, a poisoned sample is granted permanent residency.
As shown by Bonicelli et al., the model repeatedly rehearses these incorrect samples across multiple incremental steps. Karim et al. and Wu et al. demonstrate that this repetition forces the model to deeply memorize its errors, reinforcing incorrect associations and corrupting its internal representations. Instead of maintaining stability, the model continually destabilizes itself, leading to a cumulative performance collapse over time.
The Many Faces of Buffer Contamination
Buffer poisoning is not limited to explicitly flipped labels; it manifests across various sequential learning paradigms:
- Pseudo-labeling in Test-Time Adaptation: In continual test-time adaptation (CTTA), models adapt to shifting distributions on the fly using their own predictions as pseudo-labels. As highlighted by Zhu et al. and Chen et al., if a model caches samples based on false confidence, future predictions are calibrated against this corrupted data, triggering a downward spiral of cumulative errors.
- Semantic Segmentation Background Shifts: In class-incremental semantic segmentation, Kwak et al. point out that replaying old images can inadvertently overwrite historical ground-truth labels with current task background labels, causing the model to unlearn old classes.
- Synthetic Data Ingestion: In online settings flooded with generative content, Wang et al. show that storing synthetic noise instead of real-world samples rapidly degrades the buffer's utility.
- Novel Class Discovery: When discovering new categories online, Liu et al. observe that initial misclassifications contaminate the short-term memory, blinding the model to the boundaries between known and novel classes.
Mechanisms of Buffer Purification
To counteract this, recent methodologies focus on buffer purification—actively filtering, masking, or purging the memory to ensure only high-fidelity data is rehearsed.
Statistical Filtering and Self-Supervision
One approach separates clean and noisy data using the model's own behavior. For instance, Bonicelli et al. employ Gaussian Mixture Models to model loss distributions and filter out suspicious samples before they corrupt the buffer. To avoid relying entirely on noisy labels for filtering, Kim et al. introduce a self-supervised "Self-Centered filter" that leverages representation learning to maintain buffer purity independently of label correctness.
Semi-Supervised Masking
Rather than discarding noisy data entirely, Karim et al. use a masking-based purification step to segregate clean and noisy samples, treating the clean samples with standard supervised objectives and the noisy ones via semi-supervised learning techniques.
Uncertainty and Committee Voting
In test-time adaptation, where ground truth is entirely absent, filtering relies on predictive confidence. Zhu et al. design an uncertainty-aware buffer that only admits high-certainty samples. Taking this further, Chen et al. utilize a "voting committee" of text prompts in vision-language models to filter out unreliable data before it enters the dynamic cache.
Strategic Forgetting
Counterintuitively, forgetting can be leveraged as a defense mechanism. Bonicelli et al. introduce Alternate Experience Replay (AER), which alternates between learning and intentional forgetting phases. Because deep neural networks tend to forget noisy or complex patterns faster than clean, consistent patterns, this intentional forgetting phase naturally purges mislabeled data from the buffer.
Buffer Integrity Over Volume
The collective insight of these works reshapes how we think about memory in continual learning. In a clean environment, a larger buffer generally yields better performance. However, in realistic, noisy environments, the integrity of the buffer matters far more than its volume. As Wang et al. demonstrate, prioritizing a smaller, strictly verified set of real-world or clean samples allows models to maintain high performance and robust stability, even when the incoming data stream is overwhelmingly contaminated.
Key Foundational Work
Novel continual learning techniques on noisy label datasets (2023) — contributed 4 concept elements to this direction.
Continual Learning on Noisy Data Streams via Self-Purified Replay (2023) — contributed 3 concept elements to this direction.
CNLL: A Semi-supervised Approach For Continual Noisy Label Learning (2022) — contributed 1 concept elements to this direction.