When a neural network learns a sequence of tasks, it suffers from "catastrophic forgetting"—the weights optimized for Task A are overwritten to accommodate Task B. The most common defense is Experience Replay (ER): maintaining a small "replay buffer" of examples from previous tasks to periodically show the model.
However, most early implementations relied on random reservoir sampling to fill this buffer. As Daniel Brignac, Niels Lobo, and Abhijit Mahalanobis argue in Improving Replay Sample Selection and Storage for Less Forgetting in Continual Learning, random sampling is "information-blind." It treats every data point as equally valuable, often filling precious memory with redundant, easy-to-remember samples while losing the rare, difficult examples that define the boundaries of a task.
The research direction of Informed Memory Selection moves beyond random buffers. It asks: Which specific samples are most responsible for maintaining the model's internal representation of the past?
The Geometry of Decision Boundaries
One way to think about forgetting is through the geometry of the feature space. To remember a class, the model doesn't need to see every cat image it has ever processed; it only needs to remember where the "cat" region ends and the "dog" region begins.
In SupportNet, Yu Li and colleagues use Support Vector Machines (SVMs) to identify "support data"—the points closest to the decision boundaries. By storing only these critical points, the model can "review" the most informative parts of the old distribution. Similarly, Dongsub Shim and Hyunwoo Kim introduced ASER (Adversarial Shapley value Experience Replay) in Online Class-Incremental Continual Learning with Adversarial Shapley Value. They use game theory to select "sentinel" points that specifically protect the boundaries between old and new classes, ensuring that learning a new task doesn't "bleed" into the territory of an old one.
Replaying the Most Interfered Knowledge
Not all knowledge is equally at risk during a specific update. If you are learning to distinguish between different types of birds, you are more likely to forget your previous knowledge of airplanes than your knowledge of French grammar.
Rahaf Aljundi and Laurent Charlin pioneered this "targeted" approach with Maximally Interfered Retrieval (MIR) in Online Continual Learning with Maximally Interfered Retrieval. Instead of replaying random samples, MIR performs a virtual update with the new data, identifies which old samples suffer the greatest loss increase (the "interfered" samples), and replays those specifically.
This concept of "collateral damage" is also explored by Andrew Bai and Chih-Kuan Yeh in Which Pretrain Samples to Rehearse when Finetuning Pretrained Models?. Their method, mix-cd, prioritizes samples that the model previously understood but has begun to forget during fine-tuning, making the rehearsal process far more computationally efficient than uniform sampling.
Gradient Matching and Coresets
A more formal mathematical approach treats the replay buffer as a coreset: a tiny subset of data that, when trained upon, produces a gradient nearly identical to the gradient of the full dataset.
In GCR: Gradient Coreset based Replay Buffer Selection for Continual Learning, Rishabh Tiwari and Pradeep Shenoy demonstrate that you can optimize the buffer to match the "true" gradient of the past tasks. This ensures that the direction the weights move during replay is the same direction they would have moved if the model had access to all previous data. This moves replay from a heuristic "reminder" to a principled optimization constraint.
The Predictability of Forgetting
Recent work suggests that forgetting isn't a chaotic process, but a structured one. Xisen Jin and Xiang Ren show in Demystifying Language Model Forgetting with Low-rank Example Associations that forgetting in Large Language Models (LLMs) follows a "low-rank" structure. By treating forgetting like a recommendation system problem—where certain tasks "prefer" to overwrite certain types of examples—they can predict which specific pre-training data is at risk and selectively replay it.
Furthermore, Guy Hacohen and Tinne Tuytelaars discovered a "last-in-first-out" temporal pattern. In Predicting the Susceptibility of Examples to Catastrophic Forgetting, they introduce Speed-Based Sampling (SBS), which prioritizes examples that were learned at a "moderate" speed. The intuition is that examples learned too quickly are trivial, while those learned too slowly are outliers; the "moderately fast" examples represent the core structural knowledge of the task.
Why Selectivity Works
The shift from random to informed selection works because it optimizes the information density of the replay buffer. In a world of limited memory and compute:
- Boundary points (SupportNet, ASER) maintain the "shape" of classes.
- Interfered points (MIR, mix-cd) provide a surgical defense against current learning.
- Coresets (GCR) provide a mathematical proxy for the full historical gradient.
- Uncertainty/Prototypes (PMR, SS-IQA) ensure the model retains a representative "gist" of the past rather than getting lost in noise.
By treating the replay buffer as a strategic asset rather than a random bin, these methods allow models to learn indefinitely while keeping their most critical memories intact.