Locally Connected Networks for Biologically Plausible Vision

This direction is merged into another direction: weight transport problem. Its last version stays readable and citable here.

Version 1 of 121 papers23 conceptsfrozen 2026-10-04

In modern computer vision, the Convolutional Neural Network (CNN) is king. Its power stems from a single, elegant trick: weight sharing. By sliding the same small filter across an entire image, a CNN assumes that a feature useful in the top-left corner (like an edge) is equally useful in the bottom-right.

However, as Yann LeCun and Karol Gregor pointed out in Emergence of Complex-Like Cells in a Temporal Product Network with Local Receptive Fields, this assumption is a biological impossibility. In the mammalian visual cortex, neurons are distinct physical entities. There is no biological "copy-paste" mechanism that can synchronize the synaptic weights of a neuron processing the center of your vision with one processing the periphery.

This realization has sparked a research direction focused on Locally Connected Networks (LCNs)—architectures that maintain the spatial structure of convolutions but abandon the "cheat" of global weight sharing.

The Weight Sharing Paradox

Weight sharing is a double-edged sword. It provides a massive "inductive bias" that makes learning efficient, but it creates a "weight transport problem." For backpropagation to work in a CNN, the gradient must be calculated and then applied identically to every patch of the image.

As Roman Pogodin and Peter E. Latham argue in Kernelized information bottleneck leads to biologically plausible 3-factor Hebbian learning in deep networks, there is no known physiological process for synchronizing weights between distant, independent neurons. If we want to understand how the brain actually learns to see, we must find ways to train networks where every local patch has its own unique, independent parameters.

The Scaling Gap

If we simply remove weight sharing and replace it with local connections, performance usually collapses. This is known as the "scaling gap."

In Assessing the Scalability of Biologically-Motivated Deep Learning Algorithms and Architectures, Sergey Bartunov, Adam Santoro, and Geoffrey E. Hinton demonstrated that while biologically-inspired rules like Feedback Alignment work on simple tasks, they fail to scale to ImageNet when weight sharing is removed. Without the constraint of shared weights, the network has too many degrees of freedom and the learning signals become too noisy to coordinate.

Emergent Topography and Brain-Like Maps

Despite the difficulty of training them, LCNs offer something CNNs cannot: the emergence of biological structures. When a network is forced to learn locally, it develops spatial organization similar to the primate brain.

Eshed Margalit and Daniel Yamins showed in A Unifying Principle for the Functional Organization of Visual Cortex that by combining local connectivity with a spatial smoothness constraint, their model (TDANN) naturally developed "pinwheel" topographic maps in early layers and "category patches" in later layers. These features, which are hallmarks of the visual cortex, do not emerge in standard CNNs because global weight sharing washes out the local spatial variations that drive this organization.

Bridging the Gap with Local Rules

To make LCNs as effective as CNNs, researchers are looking toward more sophisticated local learning rules.

  1. Hebbian Plasticity: Dmitry Krotov and John J. Hopfield demonstrated in Local Unsupervised Learning for Image Analysis that unsupervised Hebbian learning—where neurons that "fire together, wire together"—can produce robust filters that rival backpropagation in shallow networks.
  2. Dynamic Weight Sharing: In Towards Biologically Plausible Convolutional Networks, Roman Pogodin and Timothy P Lillicrap proposed a "dynamic" version of weight sharing. Instead of an engineered constraint, they use lateral connections and sleep-like phases to allow local weights to slowly synchronize their features over time, matching CNN performance without violating biological constraints.
  3. Global Broadcasting: David G Clark and Larry Abbott explored Credit Assignment Through Broadcasting a Global Error Vector, showing that a single global neuromodulatory signal (like dopamine) can help coordinate local learning across the entire network.

By moving away from global weight sharing and toward locally connected architectures, this field is not just building better models of the brain—it is discovering how high-level intelligence can emerge from purely local, physical interactions.

Papers in this version

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_locally_connected_networks_for_biologica_51c86148,
  title = {Locally Connected Networks for Biologically Plausible Vision},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/locally-connected-networks-for-biologically-plausible-vision/rdr_51c86148729e4e71a0e0d96fbb9f59f2},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_locally_connected_networks_for_biologica_51c86148_v1,
  title = {Locally Connected Networks for Biologically Plausible Vision},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/locally-connected-networks-for-biologically-plausible-vision/rdr_51c86148729e4e71a0e0d96fbb9f59f2/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}