In online continual learning, a model must learn from a continuous stream of data without forgetting previous concepts. The standard solution is rehearsal: storing a small fraction of past data in an episodic memory buffer and replaying it alongside new samples.
However, this approach faces a fundamental bottleneck. If the buffer is small, the model quickly "memorizes" these few examples, leading to severe overfitting and poor generalization to the original distribution. Researchers are now finding that the key to unlocking the potential of limited memory is not just what we store, but how we transform it during replay.
The Bottleneck of Limited Memory
When a model sees the same few samples from a memory buffer repeatedly, it tends to converge to sharp minima in the loss landscape—solutions that work perfectly for those specific pixels but fail on the broader task. This is especially problematic in "online" settings, where the model may only see a new data batch once.
To combat this, Yaqian Zhang and colleagues introduced Repeated Augmented Rehearsal (RAR) in A simple but strong baseline for online continual learning: Repeated Augmented Rehearsal. They identified a trade-off: while multiple gradient updates on a single batch can help the model learn more from new data, it usually accelerates overfitting to the rehearsal buffer. By applying heavy stochastic data augmentation to both new and stored samples, RAR allows the model to take multiple gradient steps without "collapsing" into memorization. This process pushes the model toward flatter, more generalizable regions of the loss landscape.
Augmentation as a Force Multiplier
If rehearsal is about reminding the model of the past, augmentation is about making those reminders more informative. In Rainbow Memory: Continual Learning With a Memory of Diverse Samples, Jihwan Bang and Youngjoon Yoo demonstrate that aggressive patch-mixing (like CutMix) acts as a force multiplier for episodic memory.
In "blurry" continual learning scenarios—where task boundaries are not clearly defined and classes overlap—simply storing samples isn't enough. Rainbow Memory (RM) selects exemplars based on classification uncertainty to ensure a diverse buffer. When combined with data augmentation, these diverse samples effectively represent the underlying data distribution, preventing the model from overfitting to the specific instances stored in the buffer.
Beyond Pixels: Feature-Level Replay
The challenge of limited data isn't restricted to temporal streams; it also appears in long-tailed object detection, where rare classes are effectively "forgotten" in favor of frequent ones. Standard image-level resampling often fails here because an image containing a rare object might also contain many frequent ones, maintaining the imbalance.
In Image-Level or Object-Level? A Tale of Two Resampling Strategies for Long-Tailed Detection, Nadine Chang and Zhiding Yu propose RIO (Resampling at Image-level and Object-level). Instead of just replaying raw images, RIO uses an object-centric memory bank to replay features.
This serves as a form of implicit feature-level augmentation. Because these stored features were generated by different model snapshots (previous versions of the weights) and under different input augmentations, replaying them provides a diverse set of representations. This diversity allows the model to build robust detectors for rare classes without needing an impossibly large collection of rare images.
Why It Works: The Intuition
The success of these methods rests on a shared insight: Memory is a proxy for a distribution, not a collection of static facts.
- Diversity over Redundancy: By selecting diverse samples (as in Rainbow Memory) or object-specific features (as in RIO), we ensure the buffer covers as much of the "concept space" as possible.
- Stochasticity over Memorization: By applying heavy augmentation (as in RAR), we ensure the model never sees the exact same input twice. This forces the network to learn the underlying features of a class rather than the specific noise of a stored sample.
- Intensity over Passivity: With the protection of augmentation, models can afford to perform multiple gradient updates per batch, extracting more information from the data stream without the penalty of catastrophic forgetting.