Abstract
Compared with rate-based artificial neural networks, Spiking Neural Networks (SNN) provide a more biologically plausible model for the brain. However, how they perform supervised learning remains elusive. Inspired by recent works in equilibrium propagation, we propose a supervised learning algorithm based on Spike-Timing Dependent Plasticity (STDP) for a hierarchical SNN consisting of Leaky Integrate-and-Fire (LIF) neurons. A specific time window is designed for the presynaptic neuron, ensuring only spikes within this window participate in the STDP updating process. The model is trained on the MNIST dataset, achieving classification accuracy that approaches that of a Multilayer Perceptron (MLP) trained by standard back-propagation.
Introduction
While standard deep neural networks have revolutionized artificial intelligence, they rely on continuous, rate-based communication. In contrast, the human brain utilizes discrete, binary "spikes," which are significantly more energy-efficient. Spiking Neural Networks (SNNs) aim to emulate this biological realism. However, training SNNs is notoriously difficult because the non-differentiable nature of spikes makes standard back-propagation hard to apply directly. Biological learning is largely attributed to Spike-Timing Dependent Plasticity (STDP), where synaptic strength changes based on the millisecond-level timing between spikes. Adapting STDP for supervised tasks like image classification has remained a significant challenge in neuromorphic computing.
Methodology
The proposed framework utilizes a hierarchical SNN composed of Leaky Integrate-and-Fire (LIF) neurons. The learning process is divided into two distinct stages: an inference phase, where the network processes input normally, and a learning phase, where a supervisory target signal is applied to the output neurons to nudge their activity toward the correct label.
A core innovation of this work is the use of a sinusoidal STDP function instead of the traditional exponential decay. The authors found that this oscillatory weight-change rule yielded significantly better convergence. Furthermore, to address the "symmetry" problem in bidirectional networks—where weight updates in opposite directions can cancel each other out—the authors introduced a specific temporal window $[0, T]$. In this scheme, only presynaptic spikes within this window are considered for STDP updates, while all postsynaptic spikes are included. This asymmetry allows the network to effectively learn from the supervisory signal.
Experimental Results
The model was evaluated on the MNIST handwritten digit recognition dataset, a standard benchmark consisting of 70,000 grayscale images. The proposed SNN achieved a test accuracy of 96.8%, outperforming other STDP-based two-layer networks which typically range between 93.5% and 95.0%. This performance closely approaches that of a standard Multilayer Perceptron (MLP) trained with back-propagation (98.5%) and the equilibrium propagation (e-prop) algorithm (97.5%).
The effectiveness of the algorithm is further demonstrated by the convergence of the training error to 0.0%, indicating that the windowed STDP method can successfully minimize the loss function in a spiking environment. Figure 2 illustrates the internal dynamics of the neurons during this process.
Discussion and Conclusion
The success of this approach highlights the importance of the plasticity curve's shape and temporal precision. The authors identified an "anti-symmetry trap," where simultaneous firing rate increases lead to zero net weight change in standard STDP; this is bypassed by the temporal windowing and sinusoidal rule.
While the results on MNIST are impressive for a biologically plausible SNN, future work should explore more complex datasets like CIFAR-10 or ImageNet. Additionally, while the temporal window is a powerful computational tool, its biological counterpart remains to be identified. Integrating this STDP rule into convolutional architectures is a logical next step to further bridge the gap between biologically plausible SNNs and state-of-the-art artificial neural networks.
Summary updated