Locally Connected Layers for Position Dependent Features

You are reading version 1 of this direction, frozen 2026-10-04. The latest version is version 2.

Version 1 of 280 papers93 conceptsfrozen 2026-10-04

In the standard toolkit of deep learning, the Convolutional Neural Network (CNN) reigns supreme by assuming translation invariance. A CNN uses the same filter to detect an edge or a curve regardless of whether it appears in the top-left or bottom-right of an image. This "weight sharing" is a powerful inductive bias that reduces parameters and helps models generalize.

However, for many real-world signals, position is not an arbitrary variable—it is a defining feature. In a centered face, an eye is always above a mouth; in a brain scan, the hippocampus is always in the same anatomical neighborhood. For these structured domains, the rigid symmetry of global weight sharing becomes a limitation. Locally Connected Layers (LCLs) provide a solution by maintaining the spatial efficiency of local receptive fields while "untying" the weights, allowing every location in the input to learn its own unique transformation.

The Middle Ground of Connectivity

To understand why LCLs work, it is helpful to view them as a structural middle ground between the two extremes of neural architecture:

  1. Fully Connected Layers: Every input pixel connects to every output unit. This is maximally flexible but computationally ruinous and ignores spatial structure.
  2. Convolutional Layers: Units only look at local neighborhoods, and every neighborhood uses the exact same weights. This is efficient but assumes the world looks the same everywhere.
  3. Locally Connected Layers: Units only look at local neighborhoods, but each neighborhood has its own independent weights.

As Sergey Bartunov and Blake Aaron Richards note in Assessing the Scalability of Biologically-Motivated Deep Learning Algorithms and Architectures, this design removes the "biological implausibility" of global weight sharing while maintaining the spatial inductive bias of local connectivity. By restricting the search space to local patches but allowing those patches to specialize, LCLs capture position-specific statistics that a standard CNN would simply average away.

Specialization in Structured Domains

The primary advantage of LCLs is their ability to handle spatially registered data. When an input is aligned to a common template, the absolute coordinate of a feature becomes highly informative.

In face recognition, researchers like Xiaogang Wang and Xiaoou Tang have used LCLs in architectures like DeepID3 and Deep Learning Identity-Preserving Face Space. Because facial features like eyes, noses, and mouths appear in predictable locations, LCLs allow the network to learn specialized filters for "eye-ness" in the upper regions and "mouth-ness" in the lower regions.

This principle extends to medical imaging. Fabian Eitel and colleagues introduced Patch Individual Filter (PIF) layers in Patch individual filter layers in CNNs to harness the spatial homogeneity of neuroimaging data. They found that because brain scans are spatially normalized, learning unique filters for different brain regions allowed the model to focus on region-specific biomarkers, leading to faster training and better diagnostic accuracy.

Biological Realism and the Weight Transport Problem

Beyond engineering utility, LCLs are a cornerstone of biologically plausible deep learning. In the mammalian visual cortex, neurons in different parts of the visual field do not share identical synaptic weights. There is no biological mechanism for "copying" the weights of a neuron in the fovea to a neuron in the periphery.

Roman Pogodin and Timothy P Lillicrap argue in Towards Biologically Plausible Convolutional Networks that LCLs are a more faithful model of the brain because they avoid the "weight transport problem"—the requirement that the forward and backward passes use the exact same weight matrix. Similarly, Yann LeCun and Karol Gregor demonstrated in Emergence of Complex-Like Cells in a Temporal Product Network with Local Receptive Fields that LCLs can lead to the emergence of "pinwheel" topographic maps, mirroring the actual organization of the primary visual cortex.

Scaling and Computational Efficiency

While LCLs have many more parameters than CNNs, they are still far more efficient than fully connected networks. This sparsity makes them attractive for high-performance computing. Adam Coates and his team at Stanford/Google showed in Deep learning with COTS HPC systems that LCLs allow for the training of massive models (up to 11 billion parameters) by distributing the unique local weights across GPU clusters.

However, the parameter explosion of LCLs can lead to overfitting. To combat this, researchers have developed hybrid approaches. Xijun Wang proposed Dynamic Region-Aware Convolution (DRConv) in Dynamic Region-Aware Convolution, which dynamically partitions an image into semantic regions and applies a custom filter to each. This provides the flexibility of local specialization without the massive overhead of a unique filter for every single pixel.

Beyond Vision: Genomics and Physics

The intuition that "position matters" applies far beyond images. In GpNet: Genomic Prediction Network Using Locally Connected Layers in Korean Native Cattle, Hyo-Jun Lee used LCLs to analyze DNA sequences. Because the biological significance of a genetic marker depends on its specific chromosomal location, LCLs outperformed standard convolutions by allowing the model to learn location-specific biological interactions.

Similarly, in high-energy physics, Kevin Bauer and Peter J. Sadowski found that LCLs are superior for classifying subatomic particle jets in Jet substructure classification in high-energy physics with deep neural networks. Since the geometry of the particle detector is fixed, the spatial location of a signal is a critical physical cue that translation-invariant convolutions would ignore.

By breaking the assumption of spatial uniformity, Locally Connected Layers allow neural networks to move from "seeing" generic patterns to "understanding" the specific, structured geography of the data they process.

Papers in this version

Showing 60 of 80 papers.

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 57 papers
  2. Version 1 · 2026-10-04 · 80 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_locally_connected_layers_8ad889f9,
  title = {locally connected layers},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/locally-connected-layers/rdr_8ad889f94f844bc9b3465fe6e4c736f7},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 1, frozen text):

@misc{lacuna_locally_connected_layers_for_position_de_8ad889f9_v1,
  title = {Locally Connected Layers for Position Dependent Features},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/locally-connected-layers-for-position-dependent-features/rdr_8ad889f94f844bc9b3465fe6e4c736f7/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}