Transferring Knowledge across Learning Processes

Sebastian Flennerhag, Pablo García Moreno, Neil D. Lawrence, Andreas Damianou

Abstract

In complex transfer learning scenarios, new tasks may not be tightly linked to previous ones in parameter space. Traditional approaches that transfer information solely through final model weights often struggle with such diversity. We propose Leap, a meta-learning framework that transfers knowledge across entire learning processes. By associating each task with a manifold path from initialization to final parameters, Leap minimizes the expected length of this path. This approach leverages information obtained during training at a negligible computational cost and scales effectively from few-shot learning to demanding reinforcement learning environments like Atari.

Introduction: Beyond Weight-Space Transfer

Traditional transfer learning typically involves "finetuning" a pre-trained model on a new task, assuming that optimal parameters for different tasks are physically close in weight space. However, diverse tasks often lead to "catastrophic information loss," where the source task training discards features essential for the target task.

Furthermore, existing meta-learning methods like Model-Agnostic Meta-Learning (MAML) suffer from short-horizon bias. They are optimized for tasks requiring only a few gradient steps. When a task requires thousands or millions of steps—common in reinforcement learning—backpropagating through the entire optimization history becomes computationally prohibitive and numerically unstable. Leap addresses these gaps by focusing on the geometry of the learning process itself.

Methodology: Optimizing the Gradient Path

The core innovation of Leap is treating the learning process as a trajectory on the loss surface manifold.

Visualizing the learning process as paths on a loss surface manifold. Leap aims to find an initialization that minimizes the total path length across tasks.
  1. The Gradient Path: For any task $\tau$, the training process traces a curve $\Psi_\tau$ from an initial point $\theta^0$ to final parameters $\theta^K$.
  2. The Objective: Leap seeks an initialization $\theta^0$ that minimizes the expected "energy" or length of these paths across a distribution of tasks. The goal is to start the model where the "walk" to any solution is as short and direct as possible.
  3. The Pull-Forward Objective: To maintain efficiency, Leap uses a "pull-forward" mechanism. Instead of unrolling the entire trajectory, it "pulls" the initialization toward points along the observed training path during standard training.
  4. Computational Efficiency: By utilizing an identity approximation for Jacobians, Leap can be computed on the fly with negligible overhead, making it suitable for long-horizon tasks.

Experimental Results

The authors evaluated Leap across three distinct scenarios: standard meta-learning (Omniglot), diverse computer vision tasks (Multi-CV), and long-horizon reinforcement learning (Atari 2600).

Performance comparison of Leap against competing methods. Left: Training loss over time. Right: AUC across varying numbers of training tasks.

Key Findings

  • Omniglot: Leap outperformed Reptile and FOMAML, showing a 4 percentage point improvement in validation error over Reptile as training steps increased.
  • Multi-CV: In diverse visual domains (CIFAR, SVHN, MNIST), Leap consistently beat standard multi-headed finetuning. For instance, on CIFAR-100, Leap achieved a test error of 52.0% versus 59.2% for standard finetuning.
  • Atari: Leap is one of the few meta-learning algorithms capable of handling RL tasks involving 5 million gradient steps. Leap-initialized agents explored more effectively and reached higher scores faster than those with random initializations, demonstrating successful knowledge transfer between games with different state spaces.

Conclusion and Future Work

Leap demonstrates that the geometry of the loss surface encodes significantly more transferable information than final weights alone. By minimizing path length, the framework balances the requirements of diverse tasks into a robust meta-initialization. While currently focused on initialization, future iterations could extend this manifold-path approach to learn optimal learning rate schedules or preconditioning matrices, further enhancing the efficiency of the learning process.

Summary updated

Abstract

In complex transfer learning scenarios new tasks might not be tightly linked to previous tasks. Approaches that transfer information contained only in the final parameters of a source model will therefore struggle. Instead, transfer learning at a higher level of abstraction is needed. We propose Leap, a framework that achieves this by transferring knowledge across learning processes. We associate each task with a manifold on which the training process travels from initialization to final parameters and construct a meta-learning objective that minimizes the expected length of this path. Our framework leverages only information obtained during training and can be computed on the fly at negligible cost. We demonstrate that our framework outperforms competing methods, both in meta-learning and transfer learning, on a set of computer vision tasks. Finally, we demonstrate that Leap can transfer knowledge across learning processes in demanding reinforcement learning environments (Atari) that involve millions of gradient steps.
BibTeX
@inproceedings{flennerhag2019transferring,
  author = {Sebastian Flennerhag and Pablo García Moreno and Neil D. Lawrence and Andreas Damianou},
  title = {{Transferring Knowledge across Learning Processes}},
  booktitle = {International Conference on Learning Representations},
  year = {2019},
}
Figure