A Simple Recipe to Meta-Learn Forward and Backward Transfer

Edoardo Cetin, Antonio Carta, Oya ÇeliktutanICCV 2023 · OpenReview

Abstract

Meta-learning holds the potential to provide a general and explicit solution to tackle interference and forgetting in continual learning. However, many popular algorithms introduce expensive and unstable optimization processes with new key hyper-parameters and requirements, hindering their applicability. We propose a new, general, and simple meta-learning algorithm for continual learning (SiM4C) that explicitly optimizes to minimize forgetting and facilitate forward transfer. We show our method is stable, introduces only minimal computational overhead, and can be integrated with any memory-based continual learning algorithm in only a few lines of code. SiM4C meta-learns how to effectively continually learn even on very long task sequences, largely outperforming prior meta-approaches. Naively integrating with existing memory-based algorithms, we also record universal performance benefits and state-of-the-art results across different visual classification benchmarks without introducing new hyper-parameters.

Introduction

Deep learning models are notoriously susceptible to catastrophic forgetting during continual learning (CL)—the process of learning a sequence of tasks where new gradients often overwrite weights essential for previous tasks. While meta-learning (learning how to learn) offers a promising solution to mitigate interference, existing approaches like OML or ANML are often computationally prohibitive, unstable, and require complex meta-pre-training phases. Furthermore, these methods typically focus on preventing forgetting (backward transfer) at the expense of leveraging prior knowledge to accelerate new learning (forward transfer).

The SiM4C Algorithm

The authors propose SiM4C (Simple Meta-learning for Continual learning), a lightweight algorithm designed to optimize for both forward and backward transfer through a "simple recipe" of three core components:

  1. Single-Step Inner Loop: Unlike traditional meta-learners that unroll multiple optimization steps, SiM4C utilizes a single-step inner loop. This simplification makes the computation of second-order meta-gradients mathematically tractable and significantly more stable.
  2. "Future Data" Meta-Loss: To explicitly encourage forward transfer, the algorithm splits current task data. It performs a gradient update on one portion and evaluates the resulting model on an "unseen" portion of the same task. This objective forces the model to learn features that generalize effectively to future data within the task.
  3. Past Data Integration: The meta-loss incorporates samples from a memory buffer of previous tasks. This ensures that the single update step maintains or improves performance on historical data, directly addressing backward transfer.
The SiM4C workflow: A single inner update step is followed by a meta-loss computation on both past and future data to update the model parameters.

SiM4C is designed as a modular "plug-in" that can be integrated into existing memory-based CL algorithms, such as Experience Replay (ER) or Dark Experience Replay (DER), with minimal code modifications and no additional hyperparameters.

Experimental Results and Efficiency

SiM4C was evaluated across standard benchmarks including Omniglot, S-CIFAR-10, S-Tiny-ImageNet, and various MNIST variants (Permuted and Rotated).

Performance Gains

  • State-of-the-Art on Omniglot: SiM4C achieved over 71% accuracy, significantly outperforming ANML (~62%) and OML (~56%).
  • Universal Applicability: Integration with ER, DER, and DER++ yielded performance improvements in 35 out of 36 experimental configurations. For instance, in a challenging S-CIFAR-10 Class-IL setting with a small 200-sample buffer, SiM4C boosted standard ER accuracy from 44.79% to 54.77%.

Computational Efficiency

A key advantage of SiM4C is its efficiency. By avoiding the unrolling of multiple gradient steps, it is approximately 4 to 5 times faster than prior meta-learning methods while maintaining a significantly lower GPU memory footprint.

Comparison of optimization time and GPU memory usage between SiM4C, ANML, and OML, highlighting SiM4C's superior efficiency.

Discussion and Future Work

SiM4C demonstrates that generalization is the key to transfer; by optimizing for performance on unseen data after a single update, the model learns a representation that is "ready to learn." Unlike previous methods that suffered from meta-overfitting, SiM4C remains stable over long training durations.

While highly effective, SiM4C still relies on a memory buffer for past samples. Future research could explore ways to reduce this dependency, scale the approach to higher-resolution datasets, or apply the "simple recipe" to other domains like reinforcement learning. Additionally, further theoretical investigation into the effectiveness of single-step meta-gradients for forward transfer remains an open area for exploration.

Summary updated

Abstract

Meta-learning holds the potential to provide a general and explicit solution to tackle interference and forgetting in continual learning. However, many popular algorithms introduce expensive and unstable optimization processes with new key hyper-parameters and requirements, hindering their applicability. We propose a new, general, and simple meta-learning algorithm for continual learning (SiM4C) that explicitly optimizes to minimize forgetting and facilitate forward transfer. We show our method is stable, introduces only minimal computational overhead, and can be integrated with any memory-based continual learning algorithm in only a few lines of code. SiM4C meta-learns how to effectively continually learn even on very long task sequences, largely outperforming prior meta-approaches. Naively integrating with existing memory-based algorithms, we also record universal performance benefits and state-of-the-art results across different visual classification benchmarks without introducing new hyper-parameters.
BibTeX
@inproceedings{cetin2023simple,
  author = {Edoardo Cetin and Antonio Carta and Oya Çeliktutan},
  title = {{A Simple Recipe to Meta-Learn Forward and Backward Transfer}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2023},
  url = {https://openreview.net/forum?id=tPDAK5fjaO},
}
Figure