Abstract
Spiking Neural Networks (SNNs) offer a promising path toward energy-efficient AI, yet they remain challenging to train due to complex dynamics and high computational costs. This paper introduces a streamlined SNN training framework featuring a simplified Leaky Integrate-and-Fire (LIF) model and two novel learning rules: Efficient Multi-Spike Learning (EML) and its current-state-based variant (EMLC). By employing an event-driven computation scheme and simplifying weight update mathematics, the proposed approach enables neurons to learn multi-spike patterns with high speed and robustness to noise. Experimental results across synthetic and real-world datasets demonstrate that EML/EMLC achieves competitive accuracy while significantly outperforming existing methods in training speed and inference efficiency.
Introduction
Artificial Neural Networks (ANNs) have achieved remarkable success but are often limited by high power consumption. Spiking Neural Networks (SNNs) provide a biologically inspired alternative, communicating via discrete "spikes" that theoretically allow for massive energy savings. However, training SNNs is notoriously difficult. Existing rules often struggle with the trade-off between simplicity (deciding only if a neuron fires) and complexity (requiring precise timing for every spike). Furthermore, many models rely on fixed-time-step simulations, which are computationally wasteful. This research addresses these challenges by creating a learning framework that is both computationally efficient and capable of handling complex information encoded in spike timing and frequency.
Methodology: Simplified Neuron Model and Event-Driven Learning
The researchers introduced three key innovations to streamline SNN training:
- Simplified Neuron Model: A simplified Leaky Integrate-and-Fire (LIF) model is utilized. By replacing complex double-exponential kernels with a simpler exponential decay kernel for membrane potential, the model retains biological plausibility while reducing mathematical overhead.
- Event-Driven Scheme: Unlike traditional simulations that update at every millisecond, this algorithm only performs calculations when an input spike arrives. This ensures that computational cost scales linearly with the number of spikes rather than the simulation duration.
- Efficient Learning Rules (EML and EMLC):
- EML trains neurons to fire a specific number of spikes by simplifying the gradient descent process. It assumes that small weight changes have negligible effects on the timing of previous spikes, allowing for faster updates.
- EMLC is a faster variant that adjusts weights based on the neuron's current state (e.g., peak voltage or post-reset minimum), bypassing complex threshold calculations.
Experimental Evaluation
The methods were evaluated across several benchmarks, including synthetic spike patterns, standard UCI datasets (Breast Cancer, Iris, Zoo), and the MNIST handwritten digit dataset.
Key Insights
- Multi-Spike Versatility: A single neuron can perform multi-class classification by learning to fire a specific number of times for different categories (e.g., 5 spikes for Class A, 10 for Class B). This is significantly more efficient than the standard "one neuron per class" approach.
- Robustness to Noise: The multi-spike approach proved remarkably resilient. EML/EMLC maintained high accuracy even when 10% of input spikes were deleted or timings were jittered, conditions where simpler rules often failed.
Performance Results
- Speed: EMLC was consistently the fastest rule. On the Iris dataset, it reached target accuracy in approximately 1.08 seconds, compared to 3.17 seconds for the standard Tempotron rule.
- Accuracy: On UCI datasets, EML/EMLC achieved competitive results (e.g., ~96% on Iris). Using a hybrid CNN+EML structure, the model achieved 92.52% accuracy on MNIST, outperforming more complex spiking models with a lighter architecture.
- Efficiency: The event-driven approach enabled inference times as low as ~50 microseconds per sample, highlighting its potential for real-time processing on neuromorphic hardware.
Conclusion
This paper demonstrates that by simplifying neuron models and adopting event-driven learning rules, SNNs can achieve high performance with significantly reduced computational costs. While the current work focuses on single neurons and shallow hybrid networks, it paves the way for future implementation on specialized neuromorphic hardware like Intel's Loihi or IBM's TrueNorth. Future research will explore how these efficient rules scale to deeper, fully spiking architectures.
Summary updated