Improved Schemes for Episodic Memory-based Lifelong Learning

Yunhui Guo, Mingrui Liu, Tianbao Yang, Tajana Rosing

Abstract

Current deep neural networks can achieve remarkable performance on a single task. However, when the deep neural network is continually trained on a sequence of tasks, it seems to gradually forget the previous learned knowledge. This phenomenon is referred to as catastrophic forgetting and motivates the field called lifelong learning. Recently, episodic memory based approaches such as GEM and A-GEM have shown remarkable performance. In this paper, we provide the first unified view of episodic memory based approaches from an optimization's perspective. This view leads to two improved schemes for episodic memory based lifelong learning, called MEGA-I and MEGA-II. MEGA-I and MEGA-II modulate the balance between old tasks and the new task by integrating the current gradient with the gradient computed on the episodic memory. Notably, we show that GEM and A-GEM are degenerate cases of MEGA-I and MEGA-II which consistently put the same emphasis on the current task, regardless of how the loss changes over time. Our proposed schemes address this issue by using novel loss-balancing updating rules, which drastically improve the performance over GEM and A-GEM. Extensive experimental results show that the proposed schemes significantly advance the state-of-the-art on four commonly used lifelong learning benchmarks, reducing the error by up to 18%.

Introduction

Deep neural networks are notoriously "forgetful." While they can master a single task brilliantly, training them on a new task often causes their performance on previous tasks to plummet—a phenomenon known as catastrophic forgetting. In the field of Lifelong Learning (LLL), the goal is to create agents that learn a sequence of tasks over time, maintaining a balance between "plasticity" (learning new information) and "stability" (preserving old knowledge).

Existing methods like Gradient Episodic Memory (GEM) use a small "episodic memory" of old examples to guide new learning. However, these methods are often rigid, giving the same weight to the current task regardless of whether the model has already mastered it or is struggling. This paper introduces a unified optimization perspective to address these limitations.

Methodology: The MEGA Framework

The researchers propose two improved schemes under the MEGA (Mixed Stochastic Gradient) umbrella, which view lifelong learning as a composite objective:

  1. MEGA-I (Loss-based Balancing): This scheme adaptively weights the gradients of the current task and the memory-based tasks. If the model is performing well on the current task (low loss), it shifts focus toward reinforcing old knowledge. It uses a "sensitivity parameter" to decide when the model is starting to overfit the current task.
  2. MEGA-II (Gradient Rotation): Inspired by the geometry of optimization, this scheme rotates the current task's gradient vector to align better with the memory's gradient. It maintains the magnitude of the update required for the current task but adjusts the direction to minimize interference with past knowledge.

Mathematically, the authors demonstrate that previous standards like GEM and A-GEM are actually "degenerate" or simplified cases of the MEGA framework that lack this adaptive balancing.

Experimental Results

The researchers evaluated MEGA on four standard lifelong learning benchmarks: Permuted MNIST, Split CIFAR, Split CUB, and Split AWA. Each benchmark consists of 20 sequential tasks.

Comparison of Accuracy (At), Forgetting (Ft), and Learning Curve Area (LCA10) across different methods.

Key Performance Metrics

  • State-of-the-Art Performance: MEGA-II achieved an average accuracy of 91.21% on Permuted MNIST and 66.12% on Split CIFAR, representing significant improvements over previous SOTA methods.
  • Error Reduction: Across all benchmarks, the proposed schemes reduced the error rate by up to 18%.
  • Efficiency: Despite the increased complexity of the balancing logic, MEGA remains computationally efficient, often running faster than complex meta-learning alternatives.
Performance metrics on the Split AWA benchmark, showing accuracy, forgetting, and computational overhead.

Discussion and Insights

The core "aha moment" of this research is that task importance should not be fixed. By analyzing the ratio of current loss to memory loss, the model can dynamically prioritize the past versus the present in every mini-batch. Furthermore, MEGA-II demonstrates that the direction of learning updates matters more than intensity when data is scarce. By rotating the gradient, the model finds a path that satisfies both new and old constraints simultaneously.

Conclusion and Future Work

MEGA advances the state-of-the-art in lifelong learning by introducing adaptive, geometry-aware gradient updates. While the method still relies on episodic memory—which may pose privacy challenges in some domains—it sets a new performance ceiling. Future research may explore "task-free" continual learning and generative replay mechanisms to eliminate the need for storing raw data from previous tasks.

Summary updated

Abstract

Current deep neural networks can achieve remarkable performance on a single task. However, when the deep neural network is continually trained on a sequence of tasks, it seems to gradually forget the previous learned knowledge. This phenomenon is referred to as \textit{catastrophic forgetting} and motivates the field called lifelong learning. Recently, episodic memory based approaches such as GEM \cite{lopez2017gradient} and A-GEM \cite{chaudhry2018efficient} have shown remarkable performance. In this paper, we provide the first unified view of episodic memory based approaches from an optimization's perspective. This view leads to two improved schemes for episodic memory based lifelong learning, called MEGA-I and MEGA-II. MEGA-I and MEGA-II modulate the balance between old tasks and the new task by integrating the current gradient with the gradient computed on the episodic memory. Notably, we show that GEM and A-GEM are degenerate cases of MEGA-I and MEGA-II which consistently put the same emphasis on the current task, regardless of how the loss changes over time. Our proposed schemes address this issue by using novel loss-balancing updating rules, which drastically improve the performance over GEM and A-GEM. Extensive experimental results show that the proposed schemes significantly advance the state-of-the-art on four commonly used lifelong learning benchmarks, reducing the error by up to 18\%.
BibTeX
@inproceedings{guo2020improved,
  author = {Yunhui Guo and Mingrui Liu and Tianbao Yang and Tajana Rosing},
  title = {{Improved Schemes for Episodic Memory-based Lifelong Learning}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2020},
  url = {https://openreview.net/forum?id=yOH7CGwPSP},
}
Figure