Training Neural Networks with Spatially Local Learning Rules

225 concepts174 papers1 proposal

Standard deep learning relies heavily on backpropagation, an algorithm that distributes error signals globally across a network via a dedicated backward pass. While highly effective, backpropagation introduces significant biological and physical challenges. It requires symmetric feedback weights and forces upstream layers to possess precise knowledge of downstream weights—a constraint known as the weight transport problem.

To overcome these limitations, researchers are developing spatially local learning rules. These mechanisms restrict synaptic weight updates entirely to information available at the immediate connection point between neurons, eliminating the need for global error propagation and enabling highly efficient, decentralized learning.

The Core Mechanism: Neuron-Local Information

The fundamental intuition behind local learning is that a synapse can optimize its weight using only the signals it physically receives. Instead of relying on a global gradient computed from the final output, a local learning rule updates connection weights based strictly on the activity of adjacent pre-synaptic and post-synaptic nodes.

As noted by Pineda in Time Dependent Adaptive Neural Networks, this spatial locality bypasses the need for expensive global information distribution. This makes local rules highly scalable and ideal for implementation in analog hardware or self-learning physical processors where global communication wiring is physically constrained. Furthermore, Tavanaei et al. point out in Deep Learning in Spiking Neural Networks that updating weights using only locally available information directly resolves the weight transport problem by removing the requirement for symmetric feedback pathways.

This neuron-local constraint can be leveraged to discover highly adaptable learning rules. For instance, Metz et al. demonstrate in Learning to Learn Without Labels that by restricting update rules to pre- and post-synaptic activity, the resulting learning mechanisms become independent of the specific network architecture, allowing rules trained on small networks to generalize effectively to entirely different depths and widths. Similarly, Lindsey et al. show in Learning to Learn with Feedback and Local Plasticity that decoupling feedback connections from feedforward weights and relying on local synaptic updates allows networks to match or exceed gradient-based methods in non-stationary environments.

Hebbian Plasticity and Local Competition

Many local learning frameworks build upon Hebbian plasticity, where synaptic adjustments are driven by correlations in neural activity. In Unsupervised Learning by Competing Hidden Units, Krotov demonstrates that hidden units can develop robust feature detectors through local competition alone, relying exclusively on the activity of the connected pre- and post-synaptic neurons rather than global error signals.

To optimize more complex statistical objectives, networks can combine local Hebbian and anti-Hebbian rules. For example, Longtin shows in An online Hebbian learning rule that performs independent component analysis that incremental, local updates at individual connections are sufficient to solve blind source separation problems online. Additionally, the authors of A Neural Network for Semi-Supervised Learning on Manifolds utilize local Hebbian and anti-Hebbian updates to preserve similarity between input samples in the output space, optimizing a global manifold alignment objective without requiring backpropagation.

Local Error Minimization via Predictive Coding

How can a network perform coordinated, hierarchical learning if information is strictly local? Predictive Coding (PC) provides an elegant answer by reframing learning as a distributed error-minimization process.

Instead of propagating gradients from the top down, a predictive coding network alternates between two phases:

  1. A forward phase that generates predictions across layers.
  2. An iterative inference phase where internal neural states and weights are updated locally to minimize the discrepancies (prediction errors) between adjacent layers.

As explained by Lee et al. in Brain-inspired Predictive Coding Improves the Performance of Machine Challenging Tasks, this decoupled approach allows weights to be updated locally without a global backward pass. Achour demonstrates in $\mu$PC: Scaling Predictive Coding to 100+ Layer Networks that this iterative minimization of local prediction errors can scale successfully to networks exceeding 100 layers.

Standard Backpropagation:
[Loss] ---> (Layer 3) ---> (Layer 2) ---> (Layer 1)  -- Global Backward Pass

Predictive Coding:
(Layer 3) <-> [Local Error] <-> (Layer 2) <-> [Local Error] <-> (Layer 1)
       |                          |                          |
 [Local Update]             [Local Update]             [Local Update]

By updating synaptic weights based only on the activity of adjacent neurons to minimize these local prediction errors, networks can naturally learn complex representations. Brucklacher et al. show in Local minimization of prediction errors drives learning of invariant object representations in a generative network model of visual perception that this mechanism allows generative networks to develop view-invariant object representations from video sequences. The framework can also be extended to probabilistic models; Bogacz et al. demonstrate in Learning probability distributions of sensory inputs with Monte Carlo predictive coding that updating parameters based on local prediction errors and sampled latent states allows a network to infer full posterior distributions and learn hierarchical generative models.

Eliminating the Backward Pass via Forward and Equilibrium Dynamics

Another prominent strategy for achieving local learning is to eliminate the traditional backward pass entirely, replacing it with forward-driven or energy-based alternatives.

Forward-Driven Modulation

Rather than sending error gradients backward through the network, architectures can use error signals to modulate subsequent forward activity. In Error-driven Input Modulation: Solving the Credit Assignment Problem without a Backward Pass, the authors introduce PEPITA, a rule that calculates weight updates by comparing hidden layer activations from a standard forward pass with those from an error-modulated forward pass. Because the update depends only on the change in a layer's own activity, the learning signals remain strictly local to each layer. Similarly, Zhao et al. propose the Cascaded Forward algorithm in The Cascaded Forward Algorithm for Neural Network Training, which trains independent neural blocks and local predictors in a continuous, streaming fashion, completely removing the requirement for upstream layers to know downstream weights.

Equilibrium Propagation

Energy-based models achieve locality by treating the network as a physical system seeking equilibrium. In Equilibrium Propagation: Bridging the Gap between Energy-Based Models and Backpropagation, Scellier et al. introduce a framework where synaptic weight updates are calculated locally by comparing the activity of connected units across two distinct equilibrium states (a free state and a nudged state).

This approach bridges the gap between global optimization and local physical constraints. As Scellier later demonstrates in Quantum Equilibrium Propagation: Gradient-Descent Training of Quantum Systems, when a physical or quantum system's weights govern local interactions, the learning rules derived from energy minimization naturally remain local. This eliminates the need for complex global communication networks within hardware, providing a clear foundation for scalable, self-learning physical processors.

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 181 papers
  2. Version 1 · 2026-10-04 · 347 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_local_learning_rules_f808e121,
  title = {local learning rules},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/local-learning-rules/rdr_f808e1215c0a49aaa249e0b81fdbdffc},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 2, frozen text):

@misc{lacuna_local_learning_rules_f808e121_v2,
  title = {local learning rules},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/local-learning-rules/rdr_f808e1215c0a49aaa249e0b81fdbdffc/v/2},
  note = {Version 2, frozen 2026-10-04. Accessed 2026-11-19},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake