Improved Schemes for Episodic Memory-based Lifelong Learning

Yunhui Guo, Mingrui Liu, Tianbao Yang, Tajana Rosing

Deep neural networks are remarkably adept at mastering single tasks, but they notoriously struggle with Lifelong Learning (LLL). While humans can learn to ride a bike without forgetting how to walk, AI models suffer from catastrophic forgetting: when trained on a new task, they often overwrite the weights essential for previous tasks, causing performance on those older tasks to plummet.

To combat this, researchers have developed a new optimization framework called MEGA (Mixed Stochastic Gradient). By adaptively balancing the importance of new and old data based on current performance, MEGA prevents models from forgetting old tasks while learning new ones, reducing error rates by up to 18% compared to previous state-of-the-art methods.

The Limitations of Rigid Memory

Existing solutions to catastrophic forgetting often rely on an "episodic memory"—a small buffer of examples from past tasks used to guide the model during new training. However, popular methods like Gradient Episodic Memory (GEM) and its faster variant, A-GEM, are fundamentally rigid.

These methods treat the current task with the same level of importance regardless of whether the model is struggling with it or has already mastered it. This lack of flexibility leads to a sub-optimal balance between learning the new task (plasticity) and remembering the old ones (stability).

The MEGA Framework: A Unified Optimization Perspective

The authors introduce a unified optimization perspective that views lifelong learning as a composite objective: balancing the loss of the current task against the loss of the reference data stored in memory. They prove mathematically that previous benchmarks like GEM and A-GEM are actually "degenerate cases" of this framework, where the balancing terms are kept constant.

To fix this, they propose two dynamic schemes under the MEGA umbrella:

  1. MEGA-I (Loss-based Balancing): This scheme adaptively weights the gradients. If the model is performing well on the current task (indicated by low loss), it shifts its focus almost entirely to the episodic memory to reinforce old knowledge. If the current loss is high, it balances the two based on their relative magnitudes. By using the current loss as a signal, the model "knows" when it has the budget to focus on stability versus plasticity.
  2. MEGA-II (Gradient Rotation): Inspired by A-GEM, this version goes beyond scaling gradients by rotating the current task's gradient vector. It calculates an optimal rotation angle that aligns the update direction to satisfy both the new task and the old tasks simultaneously. The "sweet spot" for this rotation is determined by the current loss values, ensuring the model doesn't rotate too far away from a task it hasn't yet learned. This is particularly effective in data-scarce (few-shot) scenarios where gradient directions are noisy.

Key Results and Performance

The researchers evaluated MEGA across four standard lifelong learning benchmarks: Permuted MNIST, Split CIFAR, Split CUB, and Split AWA.

MEGA-I and MEGA-II consistently outperformed all baselines across these benchmarks, reducing overall error by up to 18% compared to A-GEM.

Performance on Split CIFAR

On the Split CIFAR dataset, MEGA-II achieved approximately 66% accuracy, representing a significant 5% improvement over previous state-of-the-art methods.

Performance and Efficiency on Split CUB

Furthermore, on the fine-grained Split CUB dataset, MEGA achieved a remarkable feat: it actually outperformed the "Multi-task" baseline (where the model sees all data at once), which is traditionally considered the theoretical upper bound for lifelong learning performance.

Despite the extra calculations required for gradient rotation and loss balancing, MEGA remains highly computationally efficient. It scales much better in terms of memory and time than "architecture-growing" methods like PROG-NN, which eventually run out of memory as tasks accumulate.

Limitations and Future Directions

While highly effective, MEGA does have limitations. MEGA-I relies on a sensitivity hyperparameter that determines when the model should switch its focus, adding a tuning requirement for different datasets. Additionally, like all episodic memory methods, MEGA requires storing a subset of old data. While the buffers used are small, this could still pose privacy or storage concerns in highly sensitive environments.

Future work may explore lifelong learning systems that do not require storing raw examples at all (non-exemplar-based methods), aiming to achieve MEGA's dynamic balance while further improving privacy and memory efficiency.

Summary updated

Abstract

Current deep neural networks can achieve remarkable performance on a single task. However, when the deep neural network is continually trained on a sequence of tasks, it seems to gradually forget the previous learned knowledge. This phenomenon is referred to as catastrophic forgetting and motivates the field called lifelong learning. Recently, episodic memory based approaches such as GEM and A-GEM have shown remarkable performance. In this paper, we provide the first unified view of episodic memory based approaches from an optimization's perspective. This view leads to two improved schemes for episodic memory based lifelong learning, called MEGA-\rom{1} and MEGA-\rom{2}. MEGA-\rom{1} and MEGA-\rom{2} modulate the balance between old tasks and the new task by integrating the current gradient with the gradient computed on the episodic memory. Notably, we show that GEM and A-GEM are degenerate cases of MEGA-\rom{1} and MEGA-\rom{2} which consistently put the same emphasis on the current task, regardless of how the loss changes over time. Our proposed schemes address this issue by using novel loss-balancing updating rules, which drastically improve the performance over GEM and A-GEM. Extensive experimental results show that the proposed schemes significantly advance the state-of-the-art on four commonly used lifelong learning benchmarks, reducing the error by up to 18%.
BibTeX
@inproceedings{guo2020improved,
  author = {Yunhui Guo and Mingrui Liu and Tianbao Yang and Tajana Rosing},
  title = {{Improved Schemes for Episodic Memory-based Lifelong Learning}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2020},
  url = {https://openreview.net/forum?id=yOH7CGwPSP},
}
Figure