Abstract
In complex transfer learning scenarios, new tasks may not be tightly linked to previous ones in parameter space. Traditional approaches that transfer information solely through final model weights often struggle with such diversity. We propose Leap, a meta-learning framework that transfers knowledge across entire learning processes. By associating each task with a manifold path from initialization to final parameters, Leap minimizes the expected length of this path. This approach leverages information obtained during training at a negligible computational cost and scales effectively from few-shot learning to demanding reinforcement learning environments like Atari.
Introduction: Beyond Weight-Space Transfer
Traditional transfer learning typically involves "finetuning" a pre-trained model on a new task, assuming that optimal parameters for different tasks are physically close in weight space. However, diverse tasks often lead to "catastrophic information loss," where the source task training discards features essential for the target task.
Furthermore, existing meta-learning methods like Model-Agnostic Meta-Learning (MAML) suffer from short-horizon bias. They are optimized for tasks requiring only a few gradient steps. When a task requires thousands or millions of steps—common in reinforcement learning—backpropagating through the entire optimization history becomes computationally prohibitive and numerically unstable. Leap addresses these gaps by focusing on the geometry of the learning process itself.
Methodology: Optimizing the Gradient Path
The core innovation of Leap is treating the learning process as a trajectory on the loss surface manifold.
- The Gradient Path: For any task $\tau$, the training process traces a curve $\Psi_\tau$ from an initial point $\theta^0$ to final parameters $\theta^K$.
- The Objective: Leap seeks an initialization $\theta^0$ that minimizes the expected "energy" or length of these paths across a distribution of tasks. The goal is to start the model where the "walk" to any solution is as short and direct as possible.
- The Pull-Forward Objective: To maintain efficiency, Leap uses a "pull-forward" mechanism. Instead of unrolling the entire trajectory, it "pulls" the initialization toward points along the observed training path during standard training.
- Computational Efficiency: By utilizing an identity approximation for Jacobians, Leap can be computed on the fly with negligible overhead, making it suitable for long-horizon tasks.
Experimental Results
The authors evaluated Leap across three distinct scenarios: standard meta-learning (Omniglot), diverse computer vision tasks (Multi-CV), and long-horizon reinforcement learning (Atari 2600).
Key Findings
- Omniglot: Leap outperformed Reptile and FOMAML, showing a 4 percentage point improvement in validation error over Reptile as training steps increased.
- Multi-CV: In diverse visual domains (CIFAR, SVHN, MNIST), Leap consistently beat standard multi-headed finetuning. For instance, on CIFAR-100, Leap achieved a test error of 52.0% versus 59.2% for standard finetuning.
- Atari: Leap is one of the few meta-learning algorithms capable of handling RL tasks involving 5 million gradient steps. Leap-initialized agents explored more effectively and reached higher scores faster than those with random initializations, demonstrating successful knowledge transfer between games with different state spaces.
Conclusion and Future Work
Leap demonstrates that the geometry of the loss surface encodes significantly more transferable information than final weights alone. By minimizing path length, the framework balances the requirements of diverse tasks into a robust meta-initialization. While currently focused on initialization, future iterations could extend this manifold-path approach to learn optimal learning rate schedules or preconditioning matrices, further enhancing the efficiency of the learning process.
Summary updated