Acceleration of Image Classification and Object Tracking by the Intel Neural Compute Stick 2 with Power Efficiency Evaluation on Raspberry Pi 4B

Tianyu Gao, József SütőSensors 2025 · DOI

TL;DR

This paper demonstrates how the Intel Neural Compute Stick 2 (NCS2) can dramatically accelerate deep learning tasks like image recognition and object tracking on a Raspberry Pi 4B. The study shows significant speedups (up to 1400%) and improved power efficiency, making real-time AI feasible on low-cost, embedded systems.

What Problem Does This Paper Address?

Deploying deep learning models on embedded systems, like those found in IoT devices or autonomous systems, is challenging. These systems often have limited processing power, memory, and strict power constraints. While powerful GPUs and cloud TPUs can handle complex neural networks, they are too power-hungry and expensive for many edge applications. This creates a need for efficient, low-power hardware accelerators that can boost the performance of embedded devices for real-time AI tasks such as image recognition and object tracking, without breaking the bank or draining the battery.

Methods

The researchers evaluated the Intel Neural Compute Stick 2 (NCS2) on a Raspberry Pi 4B. The NCS2 is a dedicated hardware accelerator (a VPU - Vision Processing Unit) designed to offload computationally intensive tasks from the CPU. They used the OpenVINO toolkit (version 2022.3.2) to run deep learning models.

The study focused on two main tasks:

  • Image Recognition: They used a single MobileNetV3-Small model, a lightweight architecture suitable for embedded systems.
  • Object Tracking: They employed a two-stage approach using a pre-trained person-detection model (MobileNetV2-based) and a person-reidentification model (OSNet-based) combined with the Deep SORT algorithm for tracking.

They measured inference speed (Frames Per Second - FPS) and power consumption for both the Raspberry Pi 4B's CPU alone and the Raspberry Pi 4B with the NCS2. Additionally, they investigated the impact of CPU clock frequency (overclocking and downclocking) on performance.

Datasets

The paper states that the test images for image recognition were "self-produced" and adjusted to match the model's input dimension (224x224 pixels). For object tracking, they used video streams from a USB camera. Specific public datasets were not explicitly mentioned for training or evaluation, but the models used (MobileNetV3-Small, MobileNetV2, OSNet) are standard architectures often trained on large, public datasets.

Key Insights and Ideas

  • NCS2 is a Game-Changer for Edge AI: The most significant takeaway is that the NCS2 provides a substantial performance boost for deep learning on resource-constrained devices, making previously infeasible real-time applications possible.
  • CPU Overclocking Has Limited Impact: While increasing the Raspberry Pi's CPU clock speed offers some improvement, it's marginal compared to the gains from using the NCS2. This highlights the inherent limitations of general-purpose CPUs for complex AI tasks.
  • Power Efficiency is Crucial: The NCS2 not only speeds up processing but also does so more efficiently, consuming significantly less power per frame processed, which is vital for battery-powered or energy-sensitive applications.
  • Reproducibility Matters: The authors emphasize the difficulty in setting up the OpenVINO environment for older NCS2 versions and provide a GitHub repository and detailed setup instructions to ensure their work is reproducible.

Key Results

  • Image Recognition Speedup: The NCS2 increased image recognition performance by approximately 400%, achieving over 50 FPS.
  • Object Tracking Speedup: The NCS2 accelerated real-time object tracking by 1200% to 1400%, reaching over 20 FPS.
  • Power Efficiency: The NCS2 demonstrated 200% to 400% better power efficiency per image/frame compared to the CPU alone. For person tracking, it was up to four times more power-efficient per frame.
  • CPU vs. NCS2: Even with CPU overclocking, the performance gains were minimal compared to the NCS2. The CPU alone achieved around 1.6 FPS for tracking, while the NCS2 reached ~21 FPS.

Limitations and Future Work

  • OpenVINO Version Compatibility: The paper notes that OpenVINO versions released after March 2022 no longer support the NCS2, making setup challenging for users of newer software. They provide instructions for an older, compatible version (2022.3.2).
  • Specific Models Tested: The reported speedups and efficiency gains are specific to the MobileNetV3-Small and MobileNetV2/OSNet models used. Performance might vary with different architectures.
  • Reproducibility Challenges: The authors highlight the effort required to set up the development environment, suggesting that easier installation and better documentation for older hardware/software combinations would be beneficial.

Future work could explore the NCS2's performance with a wider range of DNN models and on other embedded platforms. Further investigation into optimizing the OpenVINO environment for broader compatibility would also be valuable.

Summary updated

Abstract

This work investigates the efficiency and power consumption of using the Intel<sup>®</sup> (Santa Clara, CA, USA) Neural Compute Stick 2 (NCS2) on the Raspberry Pi 4B platform to accelerate image classification and object tracking. The motivation behind this study is to enable the real-time operation of complex neural networks in embedded systems, potentially reducing the cost of deep learning neural network deployment and expanding industrial applications. This study also supplements the OpenVINO™ 2022.3.2 documentation by recording the application of the Raspberry Pi 4B combined with the NCS2 in the latest European software repositories. Supported by OpenVINO™ 2022.3.2 and the Deep SORT algorithm, this study consists of two distinct tests: image recognition and real-time object tracking. A single model is used for image recognition, while two models are deployed for object tracking. These test cases evaluate the performance of the execution hardware by varying the different number of models in different application scenarios and evaluating the impact of NCS2 acceleration under various conditions. The results indicate that, for the specific models used in this experiment, the NCS2 increases image recognition performance by approximately 400% and real-time object tracking by around 1400% to 1200%. The results presented in this work indicate that the NCS2 can achieve more than 50 FPS (frames per second) in image recognition and more than 20 FPS in object tracking. The power efficiency obtained by using the NCS2 can vary from 200% to 400%. These findings highlight the significant performance gains NCS2 offers in constrained hardware environments.
BibTeX
@inproceedings{gao2025acceleration,
  author = {Tianyu Gao and József Sütő},
  title = {{Acceleration of Image Classification and Object Tracking by the Intel Neural Compute Stick 2 with Power Efficiency Evaluation on Raspberry Pi 4B}},
  booktitle = {Sensors},
  year = {2025},
  doi = {10.3390/s25061794},
}
Figure