Abstract
The SG ResNet and ASD modules are evaluated on multiple object recognition datasets, including the static ImageNet, CIFAR-100, CIFAR-10, and neuromorphic DVS-CIFAR10 datasets. Superior accuracy is demonstrated with a tiny simulation time step of four, specifically 94.52% top-1 accuracy on CIFAR-10 and 75.64% top-1 accuracy on CIFAR-100. Spiking RetinaNet is proposed using SG ResNet as the backbone and ASD module for information decoding as the first direct-training hybrid SNN-ANN detector for RGB images. Spiking RetinaNet with a SG ResNet34 backbone achieves an mAP of 0.296 on the object detection dataset MSCOCO.
Introduction
Spiking Neural Networks (SNNs) represent a biologically inspired paradigm for artificial intelligence, utilizing discrete binary pulses (spikes) to communicate information. This sparsity allows for extreme energy efficiency on neuromorphic hardware. However, scaling SNNs to the depth and performance levels of standard Artificial Neural Networks (ANNs) has been hindered by two primary challenges: the "Depth Problem" and the "Decoding Problem."
In traditional ANNs, Residual Networks (ResNets) utilize identity mappings to enable the training of very deep architectures. In the spiking domain, the addition of binary spikes to analog shortcuts often disrupts the spiking nature of the network or causes gradient instability. Furthermore, traditional SNN decoding relies on simple rate coding—counting spikes over time—which neglects the rich spatial and temporal information inherent in spike trains.
Methodology
Spiking Gate (SG) ResNet
To address the challenges of training deep SNNs, this research introduces the Spiking Gate (SG) ResNet. The core innovation is the Binary Selection Gate (BSG), which replaces the standard addition operation in residual blocks.
The BSG acts as a learnable switch that chooses between the identity mapping (input $X$) and the output of the spiking layers. This mechanism ensures the network remains "fully spiking" while allowing gradients to flow unimpeded through deep architectures, effectively solving the vanishing gradient problem.
Attention Spike Decoder (ASD)
To improve information extraction from spike trains, the authors developed the Attention Spike Decoder (ASD). Unlike rate coding, ASD applies a multi-dimensional attention mechanism across three axes:
- Temporal: Identifying critical time steps.
- Channel: Weighting relevant feature maps.
- Spatial: Focusing on informative regions within the image.
Spiking RetinaNet
By integrating SG ResNet as a backbone and ASD for decoding, the authors constructed Spiking RetinaNet, the first direct-training hybrid SNN-ANN detector designed for complex RGB image datasets like MSCOCO.
Experimental Results
The proposed methods were evaluated across several benchmarks, demonstrating state-of-the-art performance with minimal latency (simulation time step $T=4$).
Image Classification
On the CIFAR-10 and CIFAR-100 datasets, the models achieved top-1 accuracies of 94.52% and 75.64%, respectively. On the large-scale ImageNet dataset, SG ResNet50 reached 66.25% accuracy, surpassing previous direct-training SNN methodologies.
Object Detection
Spiking RetinaNet achieved an mAP of 0.296 on the MSCOCO dataset. While this is slightly below the performance of its pure ANN counterpart (0.319), the SNN version operates with significantly higher energy efficiency.
Energy Efficiency
The "neuromorphic dividend" is substantial: SG ResNet18 consumed 41.7 times less energy than a comparable ANN by replacing energy-intensive multiply-accumulate (MAC) operations with simple additions (ACs).
Conclusion
This work demonstrates that SNNs can achieve high performance and depth through gated identity mappings and attention-based decoding. While challenges remain—such as overfitting in larger models and the continued reliance on hybrid components for detection heads—the SG ResNet and ASD modules provide a robust framework for efficient, high-performance neuromorphic computing.
Summary updated