Abstract
Spiking Neural Networks (SNNs) offer a path toward energy-efficient neuromorphic computing by mimicking biological "spikes." However, their non-differentiable nature has historically made them difficult to train at depth. This paper presents a direct-train framework that utilizes a closed-form analytical response for non-leaky integrate-and-fire neurons. By leveraging single-spike temporal coding, the proposed model achieves state-of-the-art accuracy on CIFAR-10 and provides the first demonstration of a direct-trained SNN performing on par with traditional deep neural networks (DNNs) on the large-scale ImageNet dataset.
Introduction
SNNs are often hailed as the "third generation" of neural networks due to their potential for extreme energy efficiency on specialized hardware. Unlike standard DNNs that process continuous values, SNNs communicate via discrete temporal events. Despite their promise, training deep SNNs remains a significant challenge. Standard gradient-based optimization requires smooth, differentiable functions, whereas SNNs involve complex temporal dynamics and non-differentiable spike events. Traditionally, this necessitated slow simulations or lossy conversions from pre-trained DNNs.
Direct-Train Framework and Methods
The core innovation of this work is a direct-train framework that bypasses the need for computationally expensive membrane potential simulations during training.
Analytical Neuron Response
The researchers derived a closed-form mathematical expression for the input-output relationship of a non-leaky integrate-and-fire (IF) neuron using single-spike temporal coding (Time-To-First-Spike, or TTFS). This abstraction allows the SNN to be treated as a standard DNN during the training phase, enabling the use of standard backpropagation.
Temporal Coding and Stability
Instead of rate coding, which relies on spike frequency, the model uses the exact timing of a single spike to represent information. The authors found that non-leaky IF neurons combined with exponentially decaying waveforms provide the numerical stability required to train deep architectures like VGG16 and GoogleNet.
Hardware-Friendly Design
To demonstrate practical viability, the authors designed a neuron circuit using Phase-Domain (PD) signal processing with Voltage-Controlled Oscillators (VCOs). This approach is more energy-efficient than traditional voltage-domain circuits and naturally implements the integration required for spiking behavior.
Results and Performance
The proposed models were evaluated on MNIST, CIFAR-10, and ImageNet. Notably, this research marks the first successful report of a direct-trained SNN on the ImageNet benchmark.
Key Findings
- State-of-the-Art Accuracy: The model achieved 99.33% on MNIST and 92.68% on CIFAR-10. On ImageNet, it reached 68.8%, coming within 1% of equivalent DNN architectures.
- Efficiency and Sparsity: High "neuron sparsity" was observed; for instance, only 56% of neurons spiked during ImageNet inference. This sparsity directly correlates to lower power consumption on neuromorphic hardware.
- Robustness: The system exhibits inherent robustness to noise and hardware jitter due to its phase-domain encoding. Furthermore, the models maintain high accuracy even when weights are quantized to 4-bit precision.
Conclusion
This work bridges the performance gap between SNNs and DNNs, proving that temporal-coded spiking networks can scale to complex, large-scale tasks. While a minor performance gap remains on ImageNet, the combination of easy training via analytical responses and high hardware efficiency makes this a significant step forward for neuromorphic engineering. Future research will focus on even deeper architectures and further hardware-software optimization.
Summary updated