Efficient and Switchable CNN for Crowd Counting Based on Embedded Terminal

Jingyu Chen, Qiong Zhang, Wei-Shi Zheng, Xiaohua Xie

Abstract

Crowd counting is a critical task for urban safety and management, yet state-of-the-art deep learning models often require significant computational resources, typically necessitating server-side GPU processing. This paper introduces the Speed-Switchable CNN (SsCNN), a lightweight architecture designed for real-time crowd counting directly on embedded edge devices. The SsCNN features a unique nested design with three complexity modes—SsCNN_A, SsCNN_B, and SsCNN_C—that share parameters, allowing for dynamic on-the-fly switching between speed and accuracy. Evaluations on benchmark datasets and real-world embedded platforms demonstrate that SsCNN provides up to a 6x speedup and an 11x reduction in parameters compared to existing efficient models like MCNN, while maintaining high accuracy.

Introduction

Estimating crowd density from video feeds is essential for modern public safety and urban planning. However, the high computational demands of current models lead to several deployment challenges: high latency due to network transmission, privacy concerns from sending raw video to the cloud, and the high costs of server infrastructure.

To address these issues, there is a growing need to move computation to the "edge"—the embedded terminals such as smart cameras. These devices, however, possess limited memory and processing power. The proposed SsCNN architecture aims to bridge this gap by providing a model that is both efficient enough for edge hardware and flexible enough to adapt to varying operational requirements.

Comparison of server-based crowd counting versus the proposed embedded terminal approach.

Methods

The core innovation of this work is the Speed-Switchable CNN (SsCNN), which utilizes a "three-in-one" nested architecture.

Nested Architecture and Parameter Sharing

The model consists of three modes:

  • SsCNN_A (Tiny/Fast): The most lightweight version for maximum speed.
  • SsCNN_B (Medium): A balanced mode building upon the features of Mode A.
  • SsCNN_C (Complex/Accurate): The most robust mode, incorporating features from both A and B.

Because these modes are nested, they share the same underlying weights. This design allows for a single training process (optimizing for Mode C) and significantly reduces the storage footprint on embedded devices. Users can switch between these modes instantly based on current needs (e.g., switching to a faster mode during high-traffic periods) without retraining or reloading models.

The nested architecture of SsCNN showing the three switchable modes (A, B, and C) and the scaling branch.

Scaling Branch and Loss Function

To enhance accuracy without the overhead of deep convolutional layers, the authors introduced a Scaling Branch. This branch utilizes Global Average Pooling (GAP) to capture global context from feature maps and predict a scaling factor. This factor adjusts the final density map, correcting local counting errors with minimal computational cost.

During training, the model optimizes all three modes simultaneously using a combined loss function: $$L_{combine} = \lambda_1L_A + \lambda_2L_B + \lambda_3L_C$$ where $L_A, L_B,$ and $L_C$ represent the loss for each respective mode, ensuring that even the most lightweight version remains performant.

Experimental Results

The SsCNN was evaluated on three major benchmarks: ShanghaiTech (Parts A and B) and WorldExpo’10. Performance was also measured on actual embedded hardware, specifically the Rockchip RK3399 and the Allwinner H3.

Efficiency and Speed

On the Rockchip RK3399, SsCNN_A achieved a processing time of 0.368 seconds per image, a 6x improvement over the MCNN model (2.251 seconds). In terms of model size, SsCNN_A contains only 0.012 million parameters, which is approximately 11x fewer than MCNN and 27x fewer than a standard Fully Convolutional Network (FCN).

Accuracy

Despite its small size, the model remains highly competitive. On the ShanghaiTech Part_B dataset, SsCNN_C achieved a Mean Absolute Error (MAE) of 18.4, significantly outperforming MCNN (26.4) and even some larger specialized models. Accuracy was further bolstered by using Geometry-adaptive-σ to generate ground truth density maps, helping the model learn perspective more effectively.

Conclusion

The SsCNN provides a practical solution for deploying crowd counting on the edge. Its switchable nature allows for dynamic adaptability to real-world conditions, balancing the trade-off between speed and precision. Future work may focus on automating the mode-switching logic based on real-time power consumption or crowd volatility and further optimizing for true 30 FPS performance on high-resolution video feeds.

Summary updated

Abstract

Crowd counting plays an important role in urban management and public security. Recently, deep learning has shown a great advantage in making the quality of crowd counting more accurate. However, how to apply deep learning models to embedded terminals is still a challenging issue. The main contradiction of the problem lies in high demand for computing resources for deep learning and strict limitation of power consumption from embedded devices. In order to achieve the crowd counting by edge computing (in embedded terminals), we propose a tiny model based on convolutional neural networks. The model can be switched into other two forms to adapt tradeoff between the accuracy and efficiency. Especially, different forms share main parameters so as to save storage and avoid retraining. What is more, this model supports variant sizes of input images, which benefits the applications to a variety of embedded devices. The experiments on two different embedded terminals have shown the effectiveness, efficiency, and flexibility of our proposed model.
BibTeX
@article{chen2019efficient,
  author = {Jingyu Chen and Qiong Zhang and Wei-Shi Zheng and Xiaohua Xie},
  title = {{Efficient and Switchable CNN for Crowd Counting Based on Embedded Terminal}},
  journal = {IEEE Access},
  year = {2019},
  doi = {10.1109/access.2019.2910458},
}
Figure