Learning with Long-term Remembering: Following the Lead of Mixed Stochastic Gradient

Yunhui Guo, Mingrui Liu, Tianbao Yang, Tajana Rosing

Abstract

Current deep neural networks can achieve remarkable performance on a single task. However, when trained on a sequence of tasks, they tend to forget previously learned knowledge—a phenomenon known as catastrophic forgetting. This paper introduces MEGA (Mixed Stochastic Gradient), a unified framework for episodic memory-based lifelong learning. By adaptively balancing the gradients of new tasks with those from past experiences, MEGA-I and MEGA-II significantly outperform existing methods like GEM and A-GEM. Experimental results across four benchmarks demonstrate error reductions of up to 18%.

Introduction

The central challenge in lifelong learning is acquiring new skills without losing old ones. Episodic memory approaches attempt to solve this by storing a small buffer of past data to constrain current updates. However, existing methods like Gradient Episodic Memory (GEM) use fixed weighting schemes, failing to account for the varying difficulty or convergence states of different tasks. We propose a unified optimization perspective that treats lifelong learning as a composite objective, leading to the development of the MEGA framework.

The MEGA Framework

We introduce two adaptive schemes under the MEGA framework:

  1. MEGA-I (Loss-based Balancing): This scheme modulates the importance of current and reference gradients based on their respective loss values. If the model performs well on the current task, it shifts focus toward preserving episodic memory to prevent forgetting.
  2. MEGA-II (Gradient Rotation): This approach finds an optimal "mixed" gradient direction by rotating the update to align with both new and old task objectives, maximizing a weighted sum of cosine similarities.

Crucially, we demonstrate that previous benchmarks like GEM and A-GEM are degenerate cases of this framework that lack such adaptivity, consistently putting the same emphasis on the current task regardless of loss changes.

Experimental Evaluation

We evaluated MEGA on four conventional lifelong learning benchmarks: Permuted MNIST, Split CIFAR, Split CUB, and Split AWA (Animals with Attributes).

Performance and Knowledge Retention

MEGA-I and MEGA-II consistently achieved the highest average accuracy and the lowest "Forgetting Measure" ($F_T$). On the Split CUB benchmark, MEGA actually outperformed the "Multi-task" baseline, which was previously considered the theoretical performance ceiling for lifelong learning.

Performance metrics including accuracy ($A_T$) and forgetting ($F_T$) across Split CUB and Split AWA benchmarks.

Efficiency and Scalability

Despite the adaptive optimization steps, MEGA remains computationally efficient. In data-limited settings (e.g., 200-600 examples per task), MEGA-II proves particularly robust, leveraging gradient directionality to overcome noise that often plagues gradient magnitudes in small-batch scenarios.

Detailed metrics for the Split AWA benchmark, showing accuracy, forgetting, and resource usage.

Conclusion

The MEGA framework advances the state-of-the-art in lifelong learning by introducing adaptivity into episodic memory updates. By dynamically balancing the trade-off between new learning and old knowledge preservation, MEGA provides a robust solution to catastrophic forgetting. Future work will explore memory-free variants and applications to broader multi-task learning contexts where negative transfer is a concern.

Summary updated

Abstract

Current deep neural networks can achieve remarkable performance on a single task. However, when the deep neural network is continually trained on a sequence of tasks, it seems to gradually forget the previous learned knowledge. This phenomenon is referred to as \textit{catastrophic forgetting} and motivates the field called lifelong learning. Recently, episodic memory based approaches such as GEM \cite{lopez2017gradient} and A-GEM \cite{chaudhry2018efficient} have shown remarkable performance. In this paper, we provide the first unified view of episodic memory based approaches from an optimization's perspective. This view leads to two improved schemes for episodic memory based lifelong learning, called MEGA-I and MEGA-II. MEGA-I and MEGA-II modulate the balance between old tasks and the new task by integrating the current gradient with the gradient computed on the episodic memory. Notably, we show that GEM and A-GEM are degenerate cases of MEGA-I and MEGA-II which consistently put the same emphasis on the current task, regardless of how the loss changes over time. Our proposed schemes address this issue by using novel loss-balancing updating rules, which drastically improve the performance over GEM and A-GEM. Extensive experimental results show that the proposed schemes significantly advance the state-of-the-art on four commonly used lifelong learning benchmarks, reducing the error by up to 18\%.
BibTeX
@inproceedings{guo2020learning,
  author = {Yunhui Guo and Mingrui Liu and Tianbao Yang and Tajana Rosing},
  title = {{Learning with Long-term Remembering: Following the Lead of Mixed Stochastic Gradient}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2020},
  url = {https://openreview.net/forum?id=yOH7CGwPSP},
}
Figure