Spatial–Temporal Spiking Feature Pruning in Spiking Transformer

Zhaokun Zhou, Kaiwei Che, Jun Niu, Man Yao, Guoqi Li, Li Yuan, Guibo Luo, Yuesheng Zhu

Abstract

Spiking neural networks (SNNs) are known for brain-inspired architecture and low power consumption. Leveraging biocompatibility and self-attention mechanism, Spiking Transformers become the most promising SNN architecture with high accuracy. However, Spiking Transformers still faces the challenge of high training costs, such as a 51$M$ network requiring 181 training hours on ImageNet. In this work, we explore feature pruning to reduce training costs and overcome two challenges: high pruning ratio and lightweight pruning methods. We first analyze the spiking features and find the potential for a high pruning ratio. The majority of information is concentrated on a part of the spiking features in spiking transformer, which suggests that we can keep this part of the tokens and prune the others. To achieve lightweight, a parameter-free spatial–temporal spiking feature pruning method is proposed, which uses only a simple addition-sorting operation. The spiking features/tokens with high spike accumulation values are selected for training. The others are pruned and merged through a compensation module called Softmatch. Experimental results demonstrate that our method reduces training costs without compromising image classification accuracy. On ImageNet, our approach reduces the training time from 181 to 128 h while achieving comparable accuracy (83.13% versus 83.07%).
BibTeX
@article{zhou2025spatialtemporal,
  author = {Zhaokun Zhou and Kaiwei Che and Jun Niu and Man Yao and Guoqi Li and Li Yuan and Guibo Luo and Yuesheng Zhu},
  title = {{Spatial–Temporal Spiking Feature Pruning in Spiking Transformer}},
  journal = {IEEE Transactions on Cognitive and Developmental Systems},
  year = {2025},
  url = {https://openreview.net/forum?id=WqPlLFMHkD},
}

Loading related content...

Figure