Locally Connected Layers for Position-Dependent Features

79 concepts53 papers2 proposals

Standard Convolutional Neural Networks (CNNs) owe their immense success to a fundamental architectural assumption: spatial stationarity. By combining local receptive fields with global weight sharing, a standard convolutional layer assumes that a feature—whether it is a low-level edge or a high-level object part—maintains the same statistical meaning regardless of where it appears in an image.

However, this assumption breaks down when processing structured or globally aligned data. In tasks where absolute spatial coordinates carry critical semantic meaning, forcing a network to share weights across the entire input space can actively impede learning. Locally connected layers (also known as unshared local convolutions) solve this by decoupling local connectivity from weight sharing, allowing networks to learn region-specific features tailored to specific coordinates.


The Limits of Spatial Stationarity

To understand why locally connected layers work, we must first look at where global weight sharing fails. Consider an image of a human face that has been cropped and aligned. An eye will always appear in the upper-left and upper-right quadrants; a mouth will always appear near the bottom center.

Under a standard convolutional architecture, an eye-detecting filter is forced to scan the entire image, including the chin and forehead. This introduces two distinct inefficiencies:

  1. Statistical Dilution: The filter's weights must generalize across the distinct statistical properties of every region in the image, diluting its capacity to specialize in the fine-grained nuances of an eye.
  2. Spatial Disregard: The network loses explicit awareness of absolute position, meaning it cannot easily exploit the fact that an eye found on a chin is likely a false positive.

As demonstrated by Yaniv Taigman et al. in their landmark face verification system, DeepFace: Closing the Gap to Human-Level Performance in Face Verification, discarding global weight sharing in favor of locally connected layers allows different spatial regions to learn entirely unique feature extractors. This architectural shift ensures that the network optimizes its feature detection specifically for the expected anatomical structure at each coordinate.

This violation of spatial stationarity extends far beyond facial analysis:


Decoupling Locality from Parameter Reuse

Mechanically, a locally connected layer operates identically to a convolutional layer, with one crucial exception: the weights are untied.

If a convolutional layer applies a single $3 \times 3$ kernel across an entire $H \times W$ feature map, it requires only $3 \times 3 \times C_{in} \times C_{out}$ parameters. A locally connected layer, by contrast, allocates a completely independent $3 \times 3$ kernel to every single coordinate in the $H \times W$ grid, ballooning the parameter count by a factor of $H \times W$.

Standard Convolution:
[Kernel A] ──> Scans Position (1,1)
[Kernel A] ──> Scans Position (1,2)
[Kernel A] ──> Scans Position (1,3)

Locally Connected Layer:
[Kernel A] ──> Processes Position (1,1)
[Kernel B] ──> Processes Position (1,2)
[Kernel C] ──> Processes Position (1,3)

By isolating the benefits of local receptive fields from parameter reuse, this architecture proves a fundamental theoretical point. As eran malach showed in Computational Separation Between Convolutional and Fully-Connected Networks, the computational advantage of convolutions often stems from locality alone—the restriction of a neuron's field of view—rather than the strict enforcement of weight sharing.

Furthermore, weight sharing is not a hard requirement for achieving translation invariance. Jordan Ott et al. demonstrated in Learning in the Machine: To Share or Not to Share? that when networks with unshared local weights are trained with sufficient data augmentation, they can learn translationally invariant representations from scratch. Crucially, they do so while remaining highly sensitive to the global spatial arrangement of features—a property that standard CNNs often lose, as they tend to behave like bags of uncoordinated local features.


The Biological Imperative

Beyond engineering utility, locally connected layers bridge a massive gap between deep learning and neurobiology. Global weight sharing requires a mechanism to instantly copy and synchronize synaptic weights across millions of physically separated neurons—a feat that is biologically impossible.

In the primate visual cortex, neurons possess restricted receptive fields, but neurons processing different parts of the visual field develop distinct tuning distributions based on their specific localized input. Galen Pogoncheff et al. demonstrated this in Explaining V1 Properties with a Biologically Constrained Deep Learning Architecture, showing that replacing weight-sharing convolutions with locally connected layers yields models that align far more accurately with real neural data from the primary visual cortex (V1).

When combined with biologically plausible learning rules like Spike-Timing-Dependent Plasticity (STDP) in Spiking Neural Networks (SNNs), locally connected layers allow individual neurons to naturally specialize in features unique to their spatial context. As shown by Daniel J. Saunders et al. in Locally Connected Spiking Neural Networks for Unsupervised Feature Learning, this localized competition enables rapid, unsupervised feature extraction that matches traditional engineering techniques while remaining highly compatible with neuromorphic hardware.


Taming the Parameter Explosion

The primary drawback of locally connected layers is their parameter footprint, which scales linearly with the spatial resolution of the feature maps. To combat this, researchers have developed several hybrid architectures that strike a balance between regional specialization and parameter efficiency.

Grid-Based and Local Weight Sharing

Instead of unsharing weights at the pixel level, a model can divide a feature map into a coarser grid of patches. Weights are shared globally within each patch, but each patch maintains a unique set of parameters.

Kaili Zhao proposed this middle ground in Deep Region and Multi-label Learning for Facial Action Unit Detection, demonstrating that grid-based local weight sharing captures regional statistics in structured images without triggering an unsustainable parameter explosion. Similarly, Yi Sun et al. successfully applied local weight sharing to higher layers of convolutional networks, allowing models to preserve spatial awareness where geometric features stabilize.

Dynamic Weight Generation

Rather than hardcoding unshared parameters into the network architecture, weights can be generated on the fly. Yuval Nirkin et al. introduced this concept in HyperSeg: Patch-Wise Hypernetwork for Real-Time Semantic Segmentation, using a hypernetwork to dynamically generate distinct weights for different image patches. This allows a single model to function as an ensemble of specialized regional networks tailored to the local context of a scene.

Involutions

Another alternative is to invert the traditional design of convolution entirely. Duo Li et al. proposed this in Involution: Inverting the Inherence of Convolution for Visual Recognition, creating an operator that is spatial-specific (different weights for different locations) but channel-agnostic (shared across channels). This inversion allows for wide receptive fields and location-specific modeling while drastically reducing the parameter redundancy typically found in channel-specific weights.

Architecture Receptive Field Weights Across Space Weights Across Channels
Standard Convolution Local Shared Unshared
Locally Connected Local Unshared Unshared
Grid-Based Sharing Local Shared within patch / Unshared across patches Unshared
Involution Local / Wide Unshared Shared

Key Foundational Work

Learning to See Where and What: Training a Net to Make Saccades and Recognize Handwritten Characters (1992) — contributed 2 concept elements to this direction.
Multi-Layer Perceptrons with B-Spline Receptive Field Functions (1990) — contributed 1 concept elements to this direction.
History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 57 papers
  2. Version 1 · 2026-10-04 · 80 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_locally_connected_layers_8ad889f9,
  title = {locally connected layers},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/locally-connected-layers/rdr_8ad889f94f844bc9b3465fe6e4c736f7},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 2, frozen text):

@misc{lacuna_locally_connected_layers_8ad889f9_v2,
  title = {locally connected layers},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/locally-connected-layers/rdr_8ad889f94f844bc9b3465fe6e4c736f7/v/2},
  note = {Version 2, frozen 2026-10-04. Accessed 2026-11-21},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake