Meta Learning Backpropagation And Improving It

Louis Kirsch, Jürgen Schmidhuber

Abstract

Many concepts have been proposed for meta learning with neural networks (NNs), e.g., NNs that learn to reprogram fast weights, Hebbian plasticity, learned learning rules, and meta recurrent NNs. Our Variable Shared Meta Learning (VSML) unifies the above and demonstrates that simple weight-sharing and sparsity in an NN is sufficient to express powerful learning algorithms (LAs) in a reusable fashion. A simple implementation of VSML where the weights of a neural network are replaced by tiny LSTMs allows for implementing the backpropagation LA solely by running in forward-mode. It can even meta learn new LAs that differ from online backpropagation and generalize to datasets outside of the meta training distribution without explicit gradient calculation. Introspection reveals that our meta learned LAs learn through fast association in a way that is qualitatively different from gradient descent.

Introduction

Modern machine learning predominantly relies on backpropagation, a human-engineered algorithm for gradient-based weight updates. While highly effective, backpropagation is a fixed mechanism. The field of meta-learning seeks to enable neural networks (NNs) to discover their own learning algorithms (LAs). However, existing meta-learning approaches often suffer from overfitting—where "black-box" meta-learners memorize specific tasks—or rigidity, where they remain dependent on human-designed inner loops.

Variable Shared Meta Learning (VSML) addresses these challenges by demonstrating that simple weight-sharing and sparsity within an NN are sufficient to express powerful, reusable learning algorithms. This approach allows a system to discover LAs that are emergent, modifiable, and capable of generalizing across diverse datasets entirely through its own forward dynamics.

Variable Shared Meta Learning (VSML)

The core innovation of VSML is a fundamental shift in the representation of network weights. Instead of static scalar weights, VSML employs tiny LSTMs at every synapse.

VSML can be viewed as (a) a single RNN with a structured weight matrix, (b) a collection of many sub-RNNs, or (c) a neural network with complex neurons.

Weights as States and Massive Weight Sharing

In VSML, the "learned weight" is represented by the hidden state ($V_L$) of a synapse-level LSTM. Crucially, all synapse LSTMs share the same set of meta-parameters ($V_M$). This massive weight sharing drastically reduces the meta-parameter count—from hundreds of thousands to approximately 2,400—forcing the model to learn general rules for credit assignment rather than memorizing task-specific data.

Message Passing and Architecture

Neurons in a VSML network communicate via multi-dimensional forward and backward messages. This structure enables the system to mimic the forward and backward passes of standard NNs but utilizes learnable, non-linear dynamics.

The VSML architecture applied to MNIST, illustrating the flow of inputs, bidirectional message passing between layers, and the LSTM synapses.

The meta-parameters ($V_M$) are optimized using Evolution Strategies (ES) or backpropagation through the inner loop to minimize prediction error over time.

Key Insights and Results

VSML's design leads to several significant properties:

  • Expressivity: The authors demonstrated that VSML can "clone" backpropagation, showing that LSTM synapses can learn to store weights and compute gradients purely through recurrent dynamics.
  • Fast Association: Unlike gradient descent's incremental steps, VSML often discovers a "fast association" strategy, rapidly linking new inputs to labels upon a single exposure.
  • Generalization: A VSML model meta-trained on MNIST successfully generalized to Fashion MNIST and synthetic tasks like "Sum Sign," outperforming standard Meta RNNs which failed on out-of-distribution data.
  • Efficiency: The system is highly parameter-efficient, achieving complex learning behaviors with only ~2,400 meta-parameters. It also shows superior sample efficiency in early learning stages compared to Adam or SGD.

Limitations and Future Work

Despite its promise, VSML faces computational challenges, as replacing every weight with an LSTM increases costs to $O(W N^2)$. Additionally, the model currently exhibits a "short-horizon bias" due to meta-optimization over relatively short sequences, leading to performance plateaus in long training runs. Future research will focus on scaling VSML to larger architectures like ResNets or Transformers and exploring its application in reinforcement learning environments.

Summary updated

Abstract

Many concepts have been proposed for meta learning with neural networks (NNs), e.g., NNs that learn to reprogram fast weights, Hebbian plasticity, learned learning rules, and meta recurrent NNs. Our Variable Shared Meta Learning (VSML) unifies the above and demonstrates that simple weight-sharing and sparsity in an NN is sufficient to express powerful learning algorithms (LAs) in a reusable fashion. A simple implementation of VSML where the weights of a neural network are replaced by tiny LSTMs allows for implementing the backpropagation LA solely by running in forward-mode. It can even meta learn new LAs that differ from online backpropagation and generalize to datasets outside of the meta training distribution without explicit gradient calculation. Introspection reveals that our meta learned LAs learn through fast association in a way that is qualitatively different from gradient descent.
BibTeX
@inproceedings{kirsch2021meta,
  author = {Louis Kirsch and Jürgen Schmidhuber},
  title = {{Meta Learning Backpropagation And Improving It}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2021},
  url = {https://openreview.net/forum?id=hhU9TEvB6AF},
}
Figure