Optimizing Reusable Knowledge for Continual Learning via Metalearning

Julio Hurtado, Alain Raymond-Saez, Alvaro Soto

Artificial Neural Networks (ANNs) are notoriously bad at multitasking sequentially. When an ANN learns Task A and then moves on to Task B, the process of learning the new task usually overwrites the weights used for the first. This phenomenon, known as Catastrophic Forgetting (CF), is a major hurdle for "Continual Learning"—the ability of an AI to accumulate knowledge over time much like a human does.

Current solutions usually fall into two camps: Regularization (trying to keep weights from changing too much) or Architecture expansion (adding new neurons for every task). Regularization often fails to scale as tasks increase, while expansion can lead to "parameter explosion," where the model becomes too bulky to be practical.

To address this, researchers have introduced MARK (MetA Reusable Knowledge), a novel framework that tackles Catastrophic Forgetting by treating a neural network as a dynamic Knowledge Base. By combining task-specific trainable masks with a metalearning-based update strategy, MARK achieves state-of-the-art accuracy on complex benchmarks while using significantly fewer parameters and exhibiting almost zero forgetting.

The MARK Architecture

MARK shifts the continual learning philosophy from "mitigating overwriting" to "fostering reusability." The architecture consists of four main components:

  1. Feature Extractor ($F^t$): Provides an initial embedding of the input (e.g., a convolutional model for images).
  2. Knowledge Base (KB): A shared set of weights that acts as a repository of patterns learned across all tasks.
  3. Mask-Generating Functions ($M^t$): Task-specific modules that produce "masks." These masks act like a selective filter, telling the model which parts of the KB are relevant to the current input and task.
  4. Classifier ($C^t$): A task-specific head that produces the final prediction.
Overview of the MARK architecture, showing the feature extractor, task-specific masks, shared Knowledge Base (KB), and classifier.

The Training Secret Sauce

MARK uses a three-step training process for every new task it encounters:

  • KB Querying: The model first tries to solve the task using only the existing knowledge in the KB. It does this by training the masks and classifier while keeping the KB completely frozen.
  • KB Update (Metalearning): If the current knowledge isn't enough, the model uses an episodic metalearning strategy (inspired by the Reptile algorithm). It creates "mini-tasks" from the current data to find weight updates that are generally useful and synergistic, rather than overly specific to the current task.
  • Fine-tuning: Finally, the masks and classifier are refined using the newly updated KB.

Key Insights: The "Dwindling Update" Phenomenon

Instead of using the whole network for every task, the masks allow the model to selectively "query" the Knowledge Base. This prevents the weights needed for earlier tasks from being mangled while the model learns new ones.

One of the most interesting findings in the study was the "Dwindling Update" phenomenon. As MARK learns more tasks, the percentage of weights that need to change in the KB steadily decreases. This suggests the KB successfully accumulates "reusable" features that apply to many different scenarios, reducing the need for drastic updates later on.

The dwindling update phenomenon: as the model learns more tasks, the percentage of weights updated in the Knowledge Base decreases.

Furthermore, by using metalearning to update the KB, the model doesn't just learn how to solve the current task; it learns how to update its weights in a way that makes it easier to solve future tasks.

Key Results and Efficiency

The authors evaluated MARK on two popular benchmarks for continual learning: 20-Split CIFAR-100 and 20-Split MiniImageNet (a significantly harder test of a model's ability to generalize).

MARK demonstrated a massive leap over previous state-of-the-art methods like HAT, ACL, and GPM:

  • Accuracy: On the complex 20-Split MiniImageNet dataset, MARK outperformed the best competing method (ACL) by over 10% in average accuracy.
  • Forgetting: It achieved almost zero forgetfulness (Backward Transfer close to 0), meaning it retained its performance on Task 1 even after learning 19 subsequent tasks.
  • Efficiency: Because the KB is shared and only small masks/heads are added per task, MARK achieved these results using only 55% of the parameters and 51% of the memory required by its closest high-performing competitor, ACL.

Limitations and Future Work

Despite its success, the authors acknowledge a few hurdles:

  • Linear Scaling: While the KB is shared, the number of masks and classification heads still grows linearly with the number of tasks. In a scenario with thousands of tasks, this could eventually become a storage issue.
  • Sensitivity to the First Task: The quality of the Knowledge Base is heavily dependent on the first task the model encounters. If the first task is unrepresentative or "low quality," the KB might start on a poor foundation.
  • Replay Mechanisms: The current version of MARK does not use "memory replay" (storing old samples). Integrating replay could be a promising future direction to further boost performance.

Summary updated

Abstract

When learning tasks over time, artificial neural networks suffer from a problem known as Catastrophic Forgetting (CF). This happens when the weights of a network are overwritten during the training of a new task causing forgetting of old information. To address this issue, we propose MetA Reusable Knowledge or MARK, a new method that fosters weight reusability instead of overwriting when learning a new task. Specifically, MARK keeps a set of shared weights among tasks. We envision these shared weights as a common Knowledge Base (KB) that is not only used to learn new tasks, but also enriched with new knowledge as the model learns new tasks. Key components behind MARK are two-fold. On the one hand, a metalearning approach provides the key mechanism to incrementally enrich the KB with new knowledge and to foster weight reusability among tasks. On the other hand, a set of trainable masks provides the key mechanism to selectively choose from the KB relevant weights to solve each task. By using MARK, we achieve state of the art results in several popular benchmarks, surpassing the best performing methods in terms of average accuracy by over 10% on the 20-Split-MiniImageNet dataset, while achieving almost zero forgetfulness using 55% of the number of parameters. Furthermore, an ablation study provides evidence that, indeed, MARK is learning reusable knowledge that is selectively used by each task.
BibTeX
@inproceedings{hurtado2021optimizing,
  author = {Julio Hurtado and Alain Raymond-Saez and Alvaro Soto},
  title = {{Optimizing Reusable Knowledge for Continual Learning via Metalearning}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2021},
}
Figure