Transferring Knowledge across Learning Processes

Sebastian Flennerhag, Pablo Garcia Moreno, Neil D. Lawrence, Andreas C. Damianou

TL;DR

Standard transfer learning often struggles when source and target tasks are highly diverse because it only transfers the final model parameters. Transferring Knowledge across Learning Processes introduces Leap, a novel framework that transfers knowledge of the learning process itself. By minimizing the expected path length an optimizer travels on a loss manifold, Leap enables faster adaptation and better generalization, scaling even to complex reinforcement learning scenarios that require millions of gradient steps.

The Problem: Short-Horizon Bias in Meta-Learning

Most transfer learning relies on finetuning—taking a model trained on a source task and adjusting its parameters for a target task. This works well for similar tasks but breaks down when they differ significantly, as the model often discards information during initial training that might be vital later.

Meta-learning (learning how to learn) attempts to solve this, but popular methods like MAML (Model-Agnostic Meta-Learning) suffer from short-horizon bias. They are designed for "few-shot" learning (adapting in 1–5 steps) and become computationally explosive or unstable when a task requires thousands of gradient steps. The authors sought a way to transfer knowledge across these long-form learning processes without the massive overhead of backpropagating through the entire training history.

The Leap Framework: A Geometric Approach

The core idea behind Leap is geometric. The authors view the training of a task as a path traversed on a task manifold (the loss surface).

Training trajectories on a loss manifold

Leap aims to find an initialization point ($\theta^0$) that minimizes the expected "length" or "energy" of the path taken to reach a solution across a distribution of tasks. A shorter path means more efficient learning.

To avoid the computational cost of backpropagating through thousands of steps, Leap uses a "pull-forward" objective. It pulls the initialization towards points further along the training trajectories of previously seen tasks. This meta-objective acts like a set of springs, finding a Pareto-optimal starting point that balances the competing requirements of different tasks. Furthermore, Leap is highly lightweight; it leverages an identity approximation for Jacobians, meaning it computes updates on the fly without requiring extra forward or backward passes.

Experimental Setup & Key Results

The authors evaluated Leap across three increasingly difficult domains: Omniglot (a standard meta-learning benchmark), Multi-CV (a challenging mix of diverse vision datasets like CIFAR and SVHN), and Atari 2600 (a reinforcement learning stress test).

Comparative performance of Leap against baselines
  • Omniglot: Leap outperformed Reptile and MAML, especially as the number of training steps increased, achieving a 4-percentage-point improvement in final validation error over Reptile.
  • Multi-CV: Leap outperformed all baselines—including specialized methods like Progressive Nets and HAT—on almost every task, showing significantly faster convergence to optimal performance.
  • Atari: Leap successfully scaled to millions of steps. It consistently outperformed random initialization by finding useful exploration spaces faster and exhibiting less volatility during early RL training stages, suggesting it learns a high-level abstraction of "how to explore."

Limitations and Future Work

While powerful, Leap has a few limitations:

  • Jacobian Approximation: The method relies on an identity approximation for Jacobians ($J \approx I$). While ablation studies show this is accurate for the learning rates used, it might become a bottleneck for extremely complex architectures or very large optimization steps.
  • Heuristic Stabilizer: The authors use a heuristic "stabilizer" to ensure the meta-optimizer always follows a descent direction. More principled geometric stabilizers could be explored in the future.
  • Non-Convexity: The underlying theory assumes that different initializations will converge to points of equivalent performance. In highly non-convex landscapes where local minima vary wildly in quality, path length alone might not be a sufficient metric.

Summary updated

Abstract

In complex transfer learning scenarios new tasks might not be tightly linked to previous tasks. Approaches that transfer information contained only in the final parameters of a source model will therefore struggle. Instead, transfer learning at a higher level of abstraction is needed. We propose Leap, a framework that achieves this by transferring knowledge across learning processes. We associate each task with a manifold on which the training process travels from initialization to final parameters and construct a meta-learning objective that minimizes the expected length of this path. Our framework leverages only information obtained during training and can be computed on the fly at negligible cost. We demonstrate that our framework outperforms competing methods, both in meta-learning and transfer learning, on a set of computer vision tasks. Finally, we demonstrate that Leap can transfer knowledge across learning processes in demanding reinforcement learning environments (Atari) that involve millions of gradient steps.
BibTeX
@inproceedings{flennerhag2019transferring,
  author = {Sebastian Flennerhag and Pablo Garcia Moreno and Neil D. Lawrence and Andreas C. Damianou},
  title = {{Transferring Knowledge across Learning Processes}},
  booktitle = {International Conference on Learning Representations},
  year = {2019},
}
Figure