Abstract
Artificial neural network based Pedestrian Attribute Recognition (PAR) has been widely studied in recent years, despite many progresses, however, the energy consumption is still high. To address this issue, in this paper, we propose a Spiking Neural Network (SNN) based framework for energy-efficient attribute recognition. Specifically, we first adopt a spiking tokenizer module to transform the given pedestrian image into spiking feature representations. Then, the output will be fed into the spiking Transformer backbone networks for energy-efficient feature extraction. We feed the enhanced spiking features into a set of feed-forward networks for pedestrian attribute recognition. In addition to the widely used binary cross-entropy loss function, we also exploit knowledge distillation from the artificial neural network to the spiking Transformer network for more accurate attribute recognition. Extensive experiments on three widely used PAR benchmark datasets fully validated the effectiveness of our proposed SNN-PAR framework. The source code of this paper is released on https://github.com/Event-AHU/OpenPAR.
Introduction
Pedestrian Attribute Recognition (PAR) is a critical task in computer vision, enabling the identification of traits such as gender, age, and clothing. While deep learning models like CNNs and Transformers have achieved high accuracy, their deployment on edge devices is often hindered by high energy consumption due to intensive floating-point multiplications.
Spiking Neural Networks (SNNs) offer a biologically inspired alternative. Unlike traditional Artificial Neural Networks (ANNs), SNNs process information through discrete binary spikes. As shown in Figure 1, neurons accumulate membrane potential and fire a spike only when a threshold is reached, leading to significant energy savings on neuromorphic hardware by replacing power-intensive multiplications with simple additions.
Proposed Method: SNN-PAR
The SNN-PAR framework is designed to bridge the performance gap between SNNs and ANNs while maintaining energy efficiency. The architecture, illustrated in Figure 2, consists of several key components:
1. Spiking Tokenizer
The model begins with a spiking tokenizer that converts static pedestrian images into a temporal sequence of binary spikes. This is achieved using convolutional layers followed by Leaky Integrate-and-Fire (LIF) neurons, which transform pixel values into spike-based features.
2. Spiking Transformer Backbone
The core feature extractor is a Spiking Transformer (Spikingformer). It employs Spiking Self-Attention (SSA), which replaces standard matrix multiplications with spike-driven additions. This allows the model to capture global dependencies—such as the relationship between a person's headwear and their overall style—without the heavy computational cost of traditional attention mechanisms.
3. Dual-Level Knowledge Distillation
To improve the recognition accuracy of the SNN, a pre-trained ANN (VTB) serves as a teacher. The framework utilizes two levels of distillation:
- Feature-level KD: The SNN student learns to mimic the high-level feature representations of the ANN teacher.
- Response-level KD: The student aligns its final attribute predictions (logits) with the teacher's outputs.
Experimental Results
The SNN-PAR framework was evaluated on three major PAR benchmarks: PETA, PA100K, and RAPv1.
Performance and Efficiency
On the PA100K dataset, SNN-PAR achieved a mean Accuracy (mA) of 73.86 and an F1-score of 81.67. While there is still a small accuracy gap compared to state-of-the-art ANN models, SNN-PAR is significantly more efficient. The model contains 65.59M parameters, which is over 50% smaller than the ViT-B/16 teacher model (157.54M), making it highly suitable for memory-constrained environments.
Ablation Study
The dual-level distillation proved essential. Adding feature-level distillation alone improved the mean accuracy by +1.24%, confirming that the SNN effectively inherits sophisticated feature-extraction capabilities from the ANN teacher.
Conclusion
SNN-PAR represents the first successful application of Spiking Neural Networks to Pedestrian Attribute Recognition. By utilizing a spike-driven Transformer and a robust distillation strategy, the framework achieves a balance between accuracy and energy efficiency. Future work will focus on narrowing the accuracy gap further and exploring hybrid architectures to optimize performance for real-world security and surveillance deployments.
Summary updated