Meta Learning Backpropagation And Improving It

Louis Kirsch, Jürgen Schmidhuber

Introduction

Traditional machine learning relies heavily on human-engineered learning algorithms (LAs), with backpropagation being the undisputed workhorse. While the field of meta-learning aims to automate the discovery of these algorithms, most current approaches still depend on fixed, unmodifiable "inner-loop" components like standard gradient descent. Existing neural meta-learners typically fall into two categories: complex Fast Weight Programmers (FWPs) that explicitly update weights, or Meta RNNs that use their activations as memory but often act as "black boxes" prone to overfitting.

To address this, researchers have developed Variable Shared Meta Learning (VSML). This framework replaces the static weights of a neural network with tiny, parameter-shared LSTMs. By doing so, the network learns how to learn entirely through its own forward-mode dynamics, effectively discovering its own learning algorithms—including backpropagation—without needing hard-coded gradient descent during inference.

The VSML Framework

The core innovation of VSML is rethinking the "synapse." Instead of a single scalar weight connecting two neurons, VSML places a sub-RNN (specifically a tiny LSTM) at every connection.

VSML Architecture Views

Key technical features of this approach include:

  • Weight-Sharing & Sparsity: All of these tiny sub-RNNs share the same small set of meta-parameters ($V_M$). However, each sub-RNN maintains its own unique hidden state ($V_L$), which acts as the "learned weight."
  • Two-Mode Operation: The system can be trained in two distinct ways. It can perform Learning Algorithm Cloning, where the VSML network is forced to mimic the behavior of standard backpropagation. Alternatively, it can perform Meta-Learning from Scratch, using Evolutionary Strategies (ES) or backpropagation-through-time to optimize the LSTM parameters to minimize loss over a series of tasks.

Message Passing and Network Dynamics

Information flows through the VSML network via forward messages ($\vec{m}$) and backward messages ($\overleftarrow{m}$). This structure mimics the topology of a standard feed-forward network but allows the LSTMs to process errors and updates internally.

Message Passing in VSML

Because the meta-parameters ($V_M$) are so few (roughly 2,400) compared to the learned states ($V_L$, roughly 257,200), the system is forced through a computational bottleneck. This bottleneck prevents the network from merely memorizing specific dataset features, forcing it instead to learn general learning principles.

Key Insights and Results

The authors evaluated VSML across several image classification benchmarks (MNIST, Fashion MNIST, EMNIST, Kuzushiji MNIST) and synthetic tasks designed to test purely algorithmic learning.

  • Backpropagation is a Special Case: The authors mathematically prove that a simple RNN with weight sharing and sparsity is sufficient to implement the backpropagation algorithm.
  • Successful Cloning: VSML successfully "cloned" backpropagation, achieving 90% accuracy on MNIST and 80% on Fashion MNIST purely through the LSTM's forward-mode recurrent dynamics—without any external gradient calculations.
  • Fast Association vs. Gradient Descent: Introspection of the meta-learned LAs reveals they do not simply perform "slow" gradient descent. Instead, they exhibit "fast association." Once the network makes a mistake, it often corrects it perfectly the very next time it sees a similar input.
  • Outperforming Adam: When meta-learned from scratch, VSML learned significantly faster than online gradient descent (Adam) during the initial stages of training on new datasets. Furthermore, unlike standard Meta RNNs which failed on unseen tasks, VSML generalized effectively, proving it had learned a robust, general-purpose learning rule.

Limitations and Future Work

Despite its strong generalization, VSML currently faces computational hurdles. Running a sub-RNN for every weight is expensive, scaling at $O(W N^2)$ where $W$ is the number of weights and $N$ is the LSTM state size. This makes scaling to massive foundation models difficult without further optimization. Additionally, meta-optimization remains prone to local minima, especially as the inner loop lengthens or the architecture deepens.

Future directions include moving beyond simple classification into reinforcement learning and exploring more memory-efficient ways to implement the sub-RNN dynamics. Ultimately, VSML points toward a future where we no longer manually design credit assignment algorithms, but instead evolve them for specific hardware or task domains.

Summary updated

Abstract

Many concepts have been proposed for meta learning with neural networks (NNs), e.g., NNs that learn to reprogram fast weights, Hebbian plasticity, learned learning rules, and meta recurrent NNs. Our Variable Shared Meta Learning (VSML) unifies the above and demonstrates that simple weight-sharing and sparsity in an NN is sufficient to express powerful learning algorithms (LAs) in a reusable fashion. A simple implementation of VSML where the weights of a neural network are replaced by tiny LSTMs allows for implementing the backpropagation LA solely by running in forward-mode. It can even meta learn new LAs that differ from online backpropagation and generalize to datasets outside of the meta training distribution without explicit gradient calculation. Introspection reveals that our meta learned LAs learn through fast association in a way that is qualitatively different from gradient descent.
BibTeX
@inproceedings{kirsch2021meta,
  author = {Louis Kirsch and Jürgen Schmidhuber},
  title = {{Meta Learning Backpropagation And Improving It}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2021},
  url = {https://openreview.net/forum?id=hhU9TEvB6AF},
}
Figure