Abstract
The demand for energy-efficient, real-time perception in autonomous systems has led to increased interest in Spiking Neural Networks (SNNs). However, training SNNs for complex tasks like object detection remains challenging due to their non-differentiable nature and the sparsity of information. This paper presents Spiking CenterNet, a novel SNN architecture that performs high-efficiency object detection on event-based camera data. By combining an anchor-free detection head with a knowledge distillation (KD) framework, the model achieves a state-of-the-art 0.229 mAP on the Prophesee GEN1 dataset while consuming significantly less energy than previous spiking detectors.
Introduction
Traditional Artificial Neural Networks (ANNs) are computationally expensive, relying on continuous, high-precision floating-point operations that are ill-suited for power-constrained edge AI applications. In contrast, SNNs process information via discrete "spikes," enabling massive energy savings on specialized neuromorphic hardware. Despite their potential, SNNs often struggle with accuracy in complex vision tasks. Most existing spiking detectors rely on complex post-processing like Non-Maximum Suppression (NMS) or inefficient ANN-to-SNN conversion methods. Spiking CenterNet addresses these limitations by adopting an inherently efficient, NMS-free architecture boosted by teacher-student learning.
Spiking CenterNet Architecture
The proposed model is a fully spiking implementation of CenterNet, an anchor-free detector that identifies objects as single points (centroids) rather than fitting numerous pre-defined bounding boxes.
Design Components:
- Encoder-Decoder Backbone: The system utilizes a ResNet-18 encoder paired with a decoder inspired by M2U-Net. This configuration employs efficient upsampling and binary skip connections, which are critical for maintaining gradient flow in sparse spiking environments.
- Parametric LIF Neurons: Standard ReLU activations are replaced with Parametric Leaky Integrate-and-Fire (PLIF) neurons. These neurons include learnable parameters that allow the network to better capture the complex temporal dynamics inherent in event-based data.
- NMS-Free Detection: By leveraging CenterNet’s heatmap-based approach to locate local maxima, the model identifies objects directly. This eliminates the need for computationally expensive NMS steps common in other detectors like YOLO.
Knowledge Distillation Strategy
A key innovation of this work is the application of Knowledge Distillation (KD) to spiking object detection. The training process involves two stages: first, a standard, non-spiking ANN version of the model is trained to high accuracy; then, the SNN "student" is trained to match the output heatmaps of the ANN "teacher."
This distillation helps the SNN learn nuanced features that are typically lost in binary spike trains, effectively bridging the accuracy gap between noisy, sparse SNNs and high-precision ANNs.
Evaluation and Results
The model was evaluated on the Prophesee GEN1 Automotive Detection Dataset, which contains 39 hours of event-based recordings and 255,000 manual annotations.
Performance Metrics:
- Accuracy: Spiking CenterNet achieved a 0.229 mAP, a significant improvement over the previous state-of-the-art fully-spiking model (0.189 mAP).
- Efficiency: The model demonstrates remarkable energy efficiency, consuming between 0.62 mJ and 1.0 mJ per time step. This is less than half the energy consumption of previous leading spiking detectors.
- Sparsity: The network maintains high operational sparsity, with firing rates between 10.8% and 17.4%, ensuring that the majority of the hardware remains idle and power-efficient.
Conclusion
Spiking CenterNet represents a significant step forward in neuromorphic vision. By integrating anchor-free detection with knowledge distillation, the authors have created a model that is both accurate and exceptionally energy-efficient. While a performance gap still exists between SNNs and their ANN counterparts, this work provides a robust framework for future research in low-power, real-time object detection for robotics and autonomous vehicles.
Summary updated