Abstract
In continual learning, the learner faces a stream of data whose distribution changes over time. Modern neural networks are known to suffer under this setting, as they quickly forget previously acquired knowledge. To address such catastrophic forgetting, many continual learning methods implement different types of experience replay, re-learning on past data stored in a small buffer known as episodic memory. In this work, we complement experience replay with a new objective that we call "anchoring", where the learner uses bilevel optimization to update its knowledge on the current task, while keeping intact the predictions on some anchor points of past tasks. These anchor points are learned using gradient-based optimization to maximize forgetting, which is approximated by fine-tuning the currently trained model on the episodic memory of past tasks. Experiments on several supervised learning benchmarks for continual learning demonstrate that our approach improves the standard experience replay in terms of both accuracy and forgetting metrics and for various sizes of episodic memory.
Introduction
Artificial neural networks are notoriously poor at "continual learning"—the ability to learn a sequence of tasks sequentially. When a model learns a new task, it often overwrites the weights necessary for previous tasks, a phenomenon known as catastrophic forgetting.
Current solutions generally fall into three categories: regularization (restricting weight changes), modular approaches (adding new neurons/layers for every task), and memory-based methods (Experience Replay). While memory-based methods are currently state-of-the-art, they are limited by the quality of the samples stored in the tiny "episodic memory." If the stored samples are not representative of the decision boundaries, the model will still drift. Hindsight Anchor Learning (HAL) addresses this by identifying and protecting the most vulnerable pieces of knowledge through synthetic "anchor points."
Methodology
HAL introduces a two-step process to make memory-based learning more robust:
- Anchoring via Bilevel Optimization: Instead of standard training, HAL uses a nested optimization loop. It calculates a temporary update and then performs the final update by balancing two goals: minimizing loss on new data and ensuring the model’s predictions on specific anchor points from past tasks remain unchanged.
- Constructing Anchors in "Hindsight": To find the best anchors, HAL identifies points the model is most likely to forget. After finishing a task, the model is fine-tuned on its episodic memory to simulate future training. It then uses gradient ascent to create synthetic images (anchors) that maximize the forgetting loss.
- Mean Embedding Regularization: To prevent these synthetic anchors from becoming unrecognizable noise, a constraint keeps the anchor's features close to the average features of the actual data for that class.
Experimental Setup
The researchers evaluated HAL on four standard continual learning benchmarks: Permuted MNIST, Rotated MNIST, Split CIFAR-100, and Split miniImageNet. The experiments focused on "tiny" memory settings, storing only 1, 3, or 5 examples per class per task.
Analysis of Learned Anchors
A key insight of HAL is that the "edge" of the data distribution matters most. Visualizations of the learned anchors show that they do not simply sit in the middle of a data cluster; instead, they migrate toward the decision boundaries.
These borderline points are the first to be misclassified when a model starts to drift, making them the most effective anchors. Surprisingly, HAL performs better using one synthetic anchor per class than a model using two real data samples per class, suggesting that carefully engineered "vulnerable" points are more informative than random real-world examples.
Key Results
HAL consistently outperformed several strong baselines, including EWC, GEM, and standard Experience Replay (ER-Ring). On the Permuted MNIST benchmark, HAL achieved 73.6% accuracy, compared to 70.2% for standard experience replay.
Furthermore, HAL reduced the "forgetting metric" significantly. On Split CIFAR-100, it reduced forgetting to 0.10, compared to 0.13 for ER-Ring and 0.26 for EWC. The performance gains held steady even as the episodic memory size increased, showing that HAL complements existing memory-based strategies.
Conclusion and Future Work
While HAL provides a significant boost in performance, it does introduce computational overhead due to the bilevel optimization and gradient ascent required to build anchors. Additionally, the current version relies on known task boundaries to trigger anchor construction. Moving toward "task-free" continual learning, where boundaries are unknown, remains a primary challenge for future research. Nonetheless, HAL demonstrates that simulating future forgetting in hindsight is a powerful tool for anchoring past knowledge.
Summary updated