Reconciling meta-learning and continual learning with online mixtures of tasks

Ghassen Jerfel, Erin Grant, Tom Griffiths, Katherine A. HellerNeurIPS 2019 · OpenReview

Introduction

Standard meta-learning algorithms, such as Model-Agnostic Meta-Learning (MAML), operate on a "one-size-fits-all" assumption. They attempt to find a single set of global hyperparameters—usually a parameter initialization—that serves as a strong starting point for all tasks.

However, this approach breaks down in two major real-world scenarios:

  1. Task Heterogeneity: Transferring knowledge between highly dissimilar tasks can actually hurt performance, a phenomenon known as negative transfer. For instance, a single initialization might be a poor compromise when learning to recognize both underwater animals and aerial vehicles.
  2. Non-Stationarity (Continual Learning): In the real world, the distribution of tasks changes over time. If a meta-learner consolidates everything it knows into one set of weights, learning a new type of task often causes it to "forget" how to solve previous ones—leading to catastrophic forgetting.

To bridge the gap between meta-learning and continual learning, researchers propose treating task distributions as a mixture of hierarchical Bayesian models. By utilizing a Dirichlet Process Mixture Model (DPMM), they create a "task-agnostic" learner capable of automatically detecting when a new type of task appears, spawning a new specialized parameter initialization for it, and significantly reducing catastrophic forgetting.

Methodology

The core contribution of this work is recasting gradient-based meta-learning as a Dirichlet Process Mixture of Hierarchical Bayes models.

  • Hierarchical Bayes Perspective: The authors view MAML not merely as an optimization trick, but as an inference procedure within a hierarchical model where the global initialization acts as a prior for task-specific parameters.
  • The Mixture Model: Instead of relying on a single prior, the model uses a mixture of priors. Each "cluster" in the mixture represents a group of related tasks with its own optimized initialization.
  • Stochastic EM Algorithm: The approach employs an Expectation-Maximization (EM) strategy. In the E-step, the model calculates the likelihood of a new task belonging to existing clusters. In the M-step, it updates the cluster initializations using gradient descent.
  • Non-Parametric Growth (DPMM): To handle an unknown and changing number of task types, the model leverages a Dirichlet Process governed by a "Chinese Restaurant Process" (CRP) prior. This allows the model to be "infinite"—it can dynamically spawn a new cluster (a new initialization) if a task's likelihood is too low under all existing clusters.

Datasets and Evaluation

The researchers evaluated their approach on three main benchmarks to test both static and evolving environments:

  1. Standard miniImageNet: A common few-shot classification benchmark used to demonstrate that even in "static" settings, having multiple clusters can improve overall performance.
  2. Evolving Few-Shot Regression: A synthetic benchmark where the model is sequentially presented with three distinct types of functions: Polynomial, Sinusoid, and Sawtooth.
Sawtooth function example from the Evolving Few-Shot Regression benchmark.
  1. Evolving miniImageNet: A novel variant of the standard dataset where artistic filters are applied to images in sequential phases to simulate a shifting environment.
Examples of the Evolving miniImageNet dataset featuring plain, blur, night, and pencil filters.

Key Insights and Results

The proposed mixture model yields several significant advantages over traditional meta-learning approaches:

  • Task-Agnostic Discovery: Unlike many continual learning methods that require an "oracle" to signal when a task has changed, this model detects shifts automatically. If the current clusters cannot explain the new data, the DPMM logic naturally triggers the creation of a new cluster.
  • Modulating Transfer: By clustering, the model performs "non-uniform transfer." It shares a substantial amount of information within a cluster (positive transfer) while preventing unrelated clusters from interfering with each other (avoiding negative transfer).
  • Reduced Forgetting: Because old knowledge is "locked" in specialized clusters that aren't updated by unrelated new tasks, the model experiences graceful degradation. On the evolving miniImageNet task, the proposed method showed a catastrophic forgetting rate of only 1.12%, compared to 8.46% for MT-net and 8.83% for a uniform mixture baseline.
  • Accuracy Gains: On the standard miniImageNet 1-shot benchmark, the mixture approach achieved 51.46% accuracy (with 5 components), outperforming the standard MAML baseline (48.70%).
  • Specialization: The model successfully assigns specific clusters to specific task types (e.g., one cluster handles Polynomials, another handles Sinusoids), confirming that the latent structure of the shifting environment is captured correctly.

Limitations and Future Work

While highly effective, the current method relies on Maximum A Posteriori (MAP) point estimates for parameters. Moving toward full posterior inference could represent uncertainty more completely and further improve robustness. Additionally, maintaining multiple sets of initializations increases memory requirements as the number of clusters grows. Future work could explore how these mixture models interact with different architectures, such as those using external memory or attention mechanisms, to mitigate computational overhead.

Summary updated

Abstract

Learning-to-learn or meta-learning leverages data-driven inductive bias to increase the efficiency of learning on a novel task. This approach encounters difficulty when transfer is not advantageous, for instance, when tasks are considerably dissimilar or change over time. We use the connection between gradient-based meta-learning and hierarchical Bayes to propose a Dirichlet process mixture of hierarchical Bayesian models over the parameters of an arbitrary parametric model such as a neural network. In contrast to consolidating inductive biases into a single set of hyperparameters, our approach of task-dependent hyperparameter selection better handles latent distribution shift, as demonstrated on a set of evolving, image-based, few-shot learning benchmarks.
BibTeX
@inproceedings{jerfel2019reconciling,
  author = {Ghassen Jerfel and Erin Grant and Tom Griffiths and Katherine A. Heller},
  title = {{Reconciling meta-learning and continual learning with online mixtures of tasks}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2019},
  url = {https://openreview.net/forum?id=lICtAQG3cm},
}
Figure