Locally Connected Spiking Neural Networks for Unsupervised Feature Learning

Daniel J. Saunders, Devdhar Patel, Hananel Hazan, Hava T. Siegelmann, Robert Kozma

Abstract

Spiking Neural Networks (SNNs) represent a biologically inspired paradigm for energy-efficient computing, yet training them effectively without supervision remains a significant challenge. This paper introduces Locally Connected Spiking Neural Networks (LC-SNNs), an architecture that utilizes unsupervised spike-timing-dependent plasticity (STDP) to learn spatial features. By replacing global connectivity with local receptive fields and competitive inhibitory circuits, the model achieves 95.07% accuracy on MNIST and provides the first reported SNN results on the EMNIST Letters dataset. The LC-SNN demonstrates a 12x speedup in convergence over fully connected baselines and exhibits remarkable robustness to hardware-like synaptic failures.

Introduction

Most unsupervised SNNs rely on fully connected layers, which are parameter-heavy and difficult to scale, or convolutional layers, which often require non-biological weight-sharing mechanisms. This research addresses the gap by proposing a locally connected layer as a middle ground. By imposing a biological "inductive bias" where neurons only respond to specific regions of the input space, the network captures spatial features efficiently without the overhead of global connectivity.

Methods

Locally Connected Architecture

The core of the proposed model is a two-layer LC-SNN. Instead of every input pixel connecting to every output neuron, the input is tiled into patches (kernels) of size $k \times k$ with a stride $s$. Each patch maps to a dedicated sub-network of neurons that compete via Winner-Take-All (WTA) inhibitory circuits. This structure ensures that neurons inhibit others associated with the same input location, forcing the population to learn diverse features across the image.

Schematic of the Locally Connected Spiking Neural Network (LC-SNN) architecture showing the mapping from input patches to output neurons with local inhibition.

Neuron Model and Learning Rule

The network utilizes Leaky Integrate-and-Fire (LIF) neurons with adaptive firing thresholds to prevent single-neuron dominance. Features are learned entirely unsupervised via STDP, where synaptic weights are adjusted based on the millisecond-level timing of spikes. To turn these spikes into class labels, the authors implemented an n-gram voting mechanism that captures temporal dependencies between neuron firing sequences, providing more information than simple spike-counting methods.

Experimental Results

Performance and Data Efficiency

The LC-SNN was evaluated on MNIST and the EMNIST Letters partition. It achieved 95.07% accuracy on MNIST, matching state-of-the-art unsupervised SNNs. On the more challenging EMNIST Letters dataset, it reached 69.73%. Notably, the LC-SNN reached near-peak accuracy within just 5,000 training examples, whereas the fully connected baseline required over 60,000 examples—a 12x improvement in data efficiency.

Robustness and Graceful Degradation

The distributed nature of the locally connected architecture provides significant resilience. In stress tests, the network maintained approximately 90% accuracy even after 50% of its synapses were randomly deleted. Even with 90% synaptic removal, the model functioned at roughly 60% accuracy, highlighting its potential for deployment on neuromorphic hardware prone to manufacturing defects or power fluctuations.

Discussion

While the LC-SNN produces 2-5x more spikes than baseline models—potentially increasing power consumption—it offers a superior balance of biological realism and engineering efficiency. Unlike convolutional layers that "share" weights globally, LC-SNNs learn unique filters for every location, mimicking the physical constraints of biological neurons. Future work will focus on stacking multiple locally connected layers to learn hierarchical features and scaling the architecture to high-resolution color imagery.

Summary updated

BibTeX
@inproceedings{saunders2019locally,
  author = {Daniel J. Saunders and Devdhar Patel and Hananel Hazan and Hava T. Siegelmann and Robert Kozma},
  title = {{Locally Connected Spiking Neural Networks for Unsupervised Feature Learning}},
  booktitle = {Neural Networks},
  year = {2019},
  url = {https://openreview.net/forum?id=XHf01o09Pw},
}
Figure