Spiking-YOLO: Spiking Neural Network for Real-time Object Detection.

Sei Joon Kim, Seongsik Park, Byunggook Na, Sungroh Yoon

Introduction

Deep Neural Networks (DNNs) have revolutionized computer vision, yet their deployment on edge devices remains constrained by high power consumption and computational demands. Spiking Neural Networks (SNNs), inspired by the biological brain's event-driven communication, offer a promising path toward energy-efficient AI. Unlike traditional DNNs that process continuous values, SNNs communicate via discrete spikes, consuming power only when neurons fire. However, applying SNNs to complex tasks like object detection has historically been challenging due to the high numerical precision required for regression.

The Precision Bottleneck in SNN Conversion

While SNNs excel at simple classification tasks, object detection requires predicting precise bounding box coordinates—a regression problem. Previous attempts to convert pre-trained DNNs to SNNs often suffered from significant performance degradation. The primary culprit is the loss of information during the conversion process, particularly when using standard "layer-wise" normalization. In layer-wise normalization, weights are scaled based on the maximum activation of an entire layer. If a single channel has a very high activation while others are low, the lower-activation channels are "crushed," leading to extremely low firing rates and a loss of critical spatial information.

Methodology: Channel-wise Normalization and Signed Neurons

To address these limitations, the researchers introduced two key innovations designed to preserve precision and handle modern activation functions within an SNN framework.

Channel-wise Data-based Normalization

Instead of normalizing at the layer level, the authors proposed channel-wise normalization. By scaling each channel individually based on its own maximum activation, the network ensures that every feature map maintains a healthy firing rate. This fine-grained approach prevents the "signal starvation" seen in previous methods where low-activation channels would fail to produce enough spikes to convey information.

The DNN-to-SNN conversion process using channel-wise normalization to scale weights and preserve activation ranges across individual channels.

Signed Neuron with Imbalanced Threshold (IBT)

Modern object detectors like YOLO often utilize Leaky-ReLU activations to allow a small gradient for negative values. Standard SNN neurons typically only process positive spikes. To replicate Leaky-ReLU behavior without expensive floating-point operations, the authors developed a "signed neuron." This neuron uses a standard threshold for positive values and a separate, imbalanced negative threshold for negative values. This allows the SNN to accurately represent the "leak" using only discrete spikes, maintaining hardware efficiency while preserving the mathematical properties of the original detector.

Experimental Results

The model, dubbed Spiking-YOLO, was evaluated on the PASCAL VOC and MS COCO datasets, using a converted "Tiny YOLO" architecture as the baseline.

Accuracy and Convergence

On the PASCAL VOC benchmark, Spiking-YOLO achieved a mean Average Precision (mAP) of 51.83%, nearly matching the 53.01% mAP of the original DNN. This represents a breakthrough in SNN performance for object detection, achieving approximately 98% of the original model's accuracy. Furthermore, the channel-wise normalization significantly accelerated the network's convergence. Spiking-YOLO reaches its peak accuracy 2.3x to 4x faster in terms of simulation time steps compared to traditional layer-wise normalization methods.

Comparison of firing rates in the first convolutional layer. Channel-wise normalization (yellow) results in significantly higher and more informative firing rates across channels compared to layer-wise normalization (green).

Energy Efficiency

The most striking result is the energy gain. When simulated on neuromorphic hardware parameters (modeled after chips like IBM’s TrueNorth), Spiking-YOLO consumed approximately 280 times less energy than the original Tiny YOLO running on a standard GPU. This efficiency stems from the sparse, event-driven nature of the spiking activations, which are now better utilized thanks to the proposed normalization techniques.

Conclusion

Spiking-YOLO demonstrates that deep SNNs can achieve high-performance, real-time object detection with massive energy savings. By solving the precision issues inherent in DNN-to-SNN conversion through channel-wise normalization and novel neuron designs, this work paves the way for deploying sophisticated computer vision models on power-constrained neuromorphic hardware. Future research will likely focus on native SNN training and scaling these techniques to even more complex architectures.

Summary updated

Abstract

Over the past decade, deep neural networks (DNNs) have demonstrated remarkable performance in a variety of applications. As we try to solve more advanced problems, increasing demands for computing and power resources has become inevitable. Spiking neural networks (SNNs) have attracted widespread interest as the third-generation of neural networks due to their event-driven and low-powered nature. SNNs, however, are difficult to train, mainly owing to their complex dynamics of neurons and non-differentiable spike operations. Furthermore, their applications have been limited to relatively simple tasks such as image classification. In this study, we investigate the performance degradation of SNNs in a more challenging regression problem (i.e., object detection). Through our in-depth analysis, we introduce two novel methods: channel-wise normalization and signed neuron with imbalanced threshold, both of which provide fast and accurate information transmission for deep SNNs. Consequently, we present a first spiked-based object detection model, called Spiking-YOLO. Our experiments show that Spiking-YOLO achieves remarkable results that are comparable (up to 98%) to those of Tiny YOLO on non-trivial datasets, PASCAL VOC and MS COCO. Furthermore, Spiking-YOLO on a neuromorphic chip consumes approximately 280 times less energy than Tiny YOLO and converges 2.3 to 4 times faster than previous SNN conversion methods.
BibTeX
@inproceedings{kim2020spiking,
  author = {Sei Joon Kim and Seongsik Park and Byunggook Na and Sungroh Yoon},
  title = {{Spiking-YOLO: Spiking Neural Network for Real-time Object Detection.}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2020},
  doi = {10.1609/aaai.v34i07.6787},
}
Figure