SDSNN: A Single-Timestep Spiking Neural Network with Self-Dropping Neuron and Bayesian Optimization

Changqing Xu, Buxuan Song, Yi Liu, Xinfang Liao, Wenbin Zheng, Yintang YangarXiv 2025 · DOI · arXiv

TL;DR

This paper introduces SDSNN, a novel Spiking Neural Network (SNN) architecture that achieves high accuracy with significantly reduced latency and energy consumption by processing information in a single timestep. It achieves this through a new "Self-Dropping Neuron" mechanism and a three-stage Bayesian optimization framework for fine-tuning temporal parameters.

What Problem Does This Paper Address?

Spiking Neural Networks (SNNs) are biologically inspired models that promise significant energy efficiency due to their event-driven nature and sparse computation. However, a major bottleneck for their widespread adoption, especially in edge computing, is their reliance on multi-timestep computations. This leads to increased inference latency and energy consumption, negating some of the inherent advantages of SNNs. While existing methods like surrogate gradient training and ANN-to-SNN conversion have been explored, they often either increase computational overhead (BPTT) or require more inference timesteps for comparable accuracy. The core challenge is to achieve high performance with reduced latency and energy consumption, ideally within a single timestep.

Methods

The paper proposes two main innovations:

  • Self-Dropping Neuron (SD Neuron): This novel neuron model modifies the traditional Leaky Integrate-and-Fire (LIF) mechanism. Instead of firing solely when the membrane potential crosses a threshold, the SD neuron fires when the potential exceeds the threshold and begins to decay. This dynamic triggers a spike only when the neuron is actively processing information and about to "forget" it, aiming to increase the information content per spike. It also incorporates a mechanism to gradually lower the activation threshold over time to prevent dead neurons. Crucially, for backpropagation, gradients are only computed on the final timestep, simplifying computation.
  • Three-Stage Time Step Bayesian Search: To optimize the temporal dynamics of the SNN, the authors employ Bayesian optimization. This is not a single-shot optimization but a staged approach:
  • Stage 1 (Global Shared Step): Identifies promising time steps across all layers by evaluating a shared timestep for all layers.
  • Stage 2 (Initial Target Training Epoch): Optimizes layer-specific time steps for a shorter training epoch, focusing on rapid performance gains.
  • Stage 3 (Final Target Epoch): Further refines layer-specific time steps in the final training epoch to maximize recognition accuracy. This staged approach aims to balance exploration and exploitation of the temporal parameter space efficiently.

Datasets

The proposed SDSNN was evaluated on the following standard image classification datasets:

  • Fashion-MNIST
  • CIFAR-10
  • CIFAR-100

Key Insights and Ideas

  • Information-Rich Spikes: The Self-Dropping Neuron's mechanism of firing on potential decay, rather than just crossing a threshold, is a clever way to encode more meaningful information into each spike, potentially reducing the need for multiple timesteps.
  • Staged Optimization for Temporal Dynamics: Applying Bayesian optimization in a three-stage process, tailored to different training epochs and layer configurations, is a sophisticated approach to finding optimal temporal parameters. This acknowledges that the importance of temporal information might change throughout training.
  • Single-Timestep Efficiency: The core idea of achieving competitive accuracy with a single timestep is a significant step towards making SNNs practical for low-power, low-latency applications.

Key Results

  • Accuracy: The SDSNN achieved classification accuracies of 93.72% on Fashion-MNIST, 92.20% on CIFAR-10, and 69.45% on CIFAR-100, using only a single timestep. These accuracies are comparable to or even better than traditional multi-timestep SNNs.
  • Energy Efficiency: The SDSNN demonstrated significant energy savings: 56% on Fashion-MNIST, 21% on CIFAR-10, and 22% on CIFAR-100, compared to multi-timestep LIF models.
  • Latency Reduction: By operating in a single timestep, the inference latency is drastically reduced.
  • Memory Efficiency: The paper highlights that the single-timestep nature reduces theoretical memory complexity from O(T) to O(1), with experimental results showing a significant reduction in GPU memory usage (e.g., 0.91GB on CIFAR-10 compared to 3.00GB for BPTT methods).

Limitations and Future Work

  • Initial Timestep for SD Neuron: The paper notes that for the very first timestep, where no previous membrane potential exists, the IF (Integrate-and-Fire) method is used for spike firing in the SD neuron model. This is a minor deviation from the core SD mechanism.
  • Future Work: While not explicitly detailed, the success of this approach suggests future work could explore applying the SD neuron and staged Bayesian optimization to more complex tasks (e.g., object detection, video analysis) and larger-scale SNN architectures. Further investigation into the theoretical underpinnings of the SD neuron's information encoding capabilities could also be beneficial.

Summary updated

BibTeX
@misc{xu2025sdsnn,
  author = {Changqing Xu and Buxuan Song and Yi Liu and Xinfang Liao and Wenbin Zheng and Yintang Yang},
  title = {{SDSNN: A Single-Timestep Spiking Neural Network with Self-Dropping Neuron and Bayesian Optimization}},
  year = {2025},
  eprint = {2508.10913},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2508.10913},
}
Figure