SNN-PAR: Energy Efficient Pedestrian Attribute Recognition via Spiking Neural Networks

Haiyang Wang, Qian Zhu, Mowen She, Yabo Li, Haoyu Song, Minghe Xu, Xiao Wang

Abstract

Artificial neural network based Pedestrian Attribute Recognition (PAR) has been widely studied in recent years, despite many progresses, however, the energy consumption is still high. To address this issue, in this paper, we propose a Spiking Neural Network (SNN) based framework for energy-efficient attribute recognition. Specifically, we first adopt a spiking tokenizer module to transform the given pedestrian image into spiking feature representations. Then, the output will be fed into the spiking Transformer backbone networks for energy-efficient feature extraction. We feed the enhanced spiking features into a set of feed-forward networks for pedestrian attribute recognition. In addition to the widely used binary cross-entropy loss function, we also exploit knowledge distillation from the artificial neural network to the spiking Transformer network for more accurate attribute recognition. Extensive experiments on three widely used PAR benchmark datasets fully validated the effectiveness of our proposed SNN-PAR framework. The source code of this paper is released on https://github.com/Event-AHU/OpenPAR.

Introduction

Pedestrian Attribute Recognition (PAR) is a critical task in computer vision, enabling the identification of traits such as gender, age, and clothing. While deep learning models like CNNs and Transformers have achieved high accuracy, their deployment on edge devices is often hindered by high energy consumption due to intensive floating-point multiplications.

Spiking Neural Networks (SNNs) offer a biologically inspired alternative. Unlike traditional Artificial Neural Networks (ANNs), SNNs process information through discrete binary spikes. As shown in Figure 1, neurons accumulate membrane potential and fire a spike only when a threshold is reached, leading to significant energy savings on neuromorphic hardware by replacing power-intensive multiplications with simple additions.

Spiking neuron dynamics showing voltage accumulation and spike generation.

Proposed Method: SNN-PAR

The SNN-PAR framework is designed to bridge the performance gap between SNNs and ANNs while maintaining energy efficiency. The architecture, illustrated in Figure 2, consists of several key components:

1. Spiking Tokenizer

The model begins with a spiking tokenizer that converts static pedestrian images into a temporal sequence of binary spikes. This is achieved using convolutional layers followed by Leaky Integrate-and-Fire (LIF) neurons, which transform pixel values into spike-based features.

2. Spiking Transformer Backbone

The core feature extractor is a Spiking Transformer (Spikingformer). It employs Spiking Self-Attention (SSA), which replaces standard matrix multiplications with spike-driven additions. This allows the model to capture global dependencies—such as the relationship between a person's headwear and their overall style—without the heavy computational cost of traditional attention mechanisms.

3. Dual-Level Knowledge Distillation

To improve the recognition accuracy of the SNN, a pre-trained ANN (VTB) serves as a teacher. The framework utilizes two levels of distillation:

  • Feature-level KD: The SNN student learns to mimic the high-level feature representations of the ANN teacher.
  • Response-level KD: The student aligns its final attribute predictions (logits) with the teacher's outputs.
The SNN-PAR framework architecture featuring a teacher-student distillation setup.

Experimental Results

The SNN-PAR framework was evaluated on three major PAR benchmarks: PETA, PA100K, and RAPv1.

Performance and Efficiency

On the PA100K dataset, SNN-PAR achieved a mean Accuracy (mA) of 73.86 and an F1-score of 81.67. While there is still a small accuracy gap compared to state-of-the-art ANN models, SNN-PAR is significantly more efficient. The model contains 65.59M parameters, which is over 50% smaller than the ViT-B/16 teacher model (157.54M), making it highly suitable for memory-constrained environments.

Ablation Study

The dual-level distillation proved essential. Adding feature-level distillation alone improved the mean accuracy by +1.24%, confirming that the SNN effectively inherits sophisticated feature-extraction capabilities from the ANN teacher.

Conclusion

SNN-PAR represents the first successful application of Spiking Neural Networks to Pedestrian Attribute Recognition. By utilizing a spike-driven Transformer and a robust distillation strategy, the framework achieves a balance between accuracy and energy efficiency. Future work will focus on narrowing the accuracy gap further and exploring hybrid architectures to optimize performance for real-world security and surveillance deployments.

Summary updated

Abstract

Artificial neural network based Pedestrian Attribute Recognition (PAR) has been widely studied in recent years, despite many progresses, however, the energy consumption is still high. To address this issue, in this paper, we propose a Spiking Neural Network (SNN) based framework for energy-efficient attribute recognition. Specifically, we first adopt a spiking tokenizer module to transform the given pedestrian image into spiking feature representations. Then, the output will be fed into the spiking Transformer backbone networks for energy-efficient feature extraction. We feed the enhanced spiking features into a set of feed-forward networks for pedestrian attribute recognition. In addition to the widely used binary cross-entropy loss function, we also exploit knowledge distillation from the artificial neural network to the spiking Transformer network for more accurate attribute recognition. Extensive experiments on three widely used PAR benchmark datasets fully validated the effectiveness of our proposed SNN-PAR framework. The source code of this paper is released on \url{https://github.com/Event-AHU/OpenPAR}.
BibTeX
@misc{wang2024snn,
  author = {Haiyang Wang and Qian Zhu and Mowen She and Yabo Li and Haoyu Song and Minghe Xu and Xiao Wang},
  title = {{SNN-PAR: Energy Efficient Pedestrian Attribute Recognition via Spiking Neural Networks}},
  year = {2024},
  eprint = {2410.07857},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2410.07857},
}
Figure