Abstract
Spikes are the currency in central nervous systems for information transmission and processing. They are also believed to play an essential role in low-power consumption of the biological systems, whose efficiency attracts increasing attentions to the field of neuromorphic computing. However, efficient processing and learning of discrete spikes still remains as a challenging problem. In this paper, we make our contributions towards this direction. A simplified spiking neuron model is firstly introduced with effects of both synaptic input and firing output on membrane potential being modeled with an impulse function. An event-driven scheme is then presented to further improve the processing efficiency. Based on the neuron model, we propose two new multi-spike learning rules which demonstrate better performance over other baselines on various tasks including association, classification, feature detection. In addition to efficiency, our learning rules demonstrate a high robustness against strong noise of different types. They can also be generalized to different spike coding schemes for the classification task, and notably single neuron is capable of solving multi-category classifications with our learning rules. In the feature detection task, we re-examine the ability of unsupervised STDP with its limitations being presented, and find a new phenomenon of losing selectivity. In contrast, our proposed learning rules can reliably solve the task over a wide range of conditions without specific constraints being applied. Moreover, our rules can not only detect features but also discriminate them. The improved performance of our methods would contribute to neuromorphic computing as a preferable choice.
Introduction
While traditional Artificial Neural Networks (ANNs) have achieved massive success, they are notoriously power-hungry, often requiring massive GPU clusters. In contrast, the human brain operates on a tiny energy budget, communicating via discrete electrical pulses called spikes. Neuromorphic computing aims to mimic this efficiency using Spiking Neural Networks (SNNs). However, training SNNs is difficult because spikes are discrete events, making standard gradient-based optimization mathematically complex. Existing multi-spike learning rules are often computationally expensive or restricted to specific coding schemes. This research aims to bridge this gap by creating learning rules that are biologically plausible, mathematically simple, and fast enough for low-power hardware.
Methodology
Simplified Neuron Model and Event-Driven Scheme
The researchers propose a streamlined Leaky Integrate-and-Fire (LIF) model where both inputs and firing resets are treated as impulse functions. This allows the model to be converted into a Spike Response Model (SRM) form that is easier to analyze mathematically. To further improve efficiency, an event-driven scheme is employed: instead of simulating every millisecond of time (clock-based), the system only performs calculations when a spike actually occurs, significantly reducing the number of operations.
Novel Learning Rules: EML and EMLC
Two new supervised learning rules are introduced:
- EML (Efficient Multi-Spike Learning): Based on the "Spike-Threshold-Surface," this rule simplifies calculations by neglecting the complex history of previous output spikes when calculating derivatives.
- EMLC (EML-Current): A heuristic approach that modifies weights based only on the neuron's current state (e.g., the maximum voltage reached or the minimum reset potential). This avoids complex derivative calculations entirely, making it extremely fast.
Experimental Results
Performance and Speed
The proposed EMLC rule is approximately 10x faster than the Multi-Spike Tempotron (MST) in terms of CPU execution time during training. In complex feature detection tasks, both EML and EMLC are roughly 2x faster than other baseline rules like TDP.
Robustness and Generalization
The learning rules maintained 100% accuracy even when spike timings were "jittered" by 100ms or when 40% of spikes were randomly deleted from the input. Furthermore, the rules work equally well for both temporal coding (timing-based) and rate coding (frequency-based) schemes.
Discussion and Insights
The "Selectivity Loss" in STDP
The authors re-examined Spike-Timing-Dependent Plasticity (STDP), a popular unsupervised learning rule. They discovered that while STDP is good at initially finding features, it eventually "over-learns," causing the neuron to lose its selectivity and stop firing altogether due to the dominance of long-term depression (LTD). In contrast, the proposed rules can reliably solve feature detection over a wide range of conditions.
Single-Neuron Multi-Classing
Traditionally, one neuron might represent one class. This study demonstrates that a single neuron can be trained to respond to different categories with different numbers of spikes (e.g., 5 spikes for Class A, 10 for Class B), effectively increasing the information density of a single cell.
Conclusion
This work introduces efficient approaches for multi-spike learning that offer significant speedups and high robustness to noise. By simplifying the neuron model and ignoring unnecessary history in derivative calculations, the authors provide a framework suitable for low-power neuromorphic hardware. Future work will focus on extending these single-layer rules to deep, multi-layered SNN architectures and testing them on actual neuromorphic chips.
Summary updated