Fast and Accurate Convolutional Object Detectors for Real-time Embedded Platforms

Min-Kook Choi, Jaehyung Park, Heechul Jung, Jin-Hee Lee, Soo-Heang Eo

Abstract

Optimizing state-of-the-art object detectors for high-end GPUs often leads to sub-optimal performance on edge devices. This study investigates the performance gap between desktop GPUs and embedded platforms like the NVIDIA Jetson Xavier and Drive PX2. By redesigning the RefineDet architecture and optimizing post-processing bottlenecks, the authors achieve real-time performance on embedded hardware without significant accuracy loss.

Introduction

Computer vision benchmarks are typically dominated by models optimized for massive desktop GPUs. However, deploying these models on edge devices reveals a disconnect: the fastest model on a Titan XP is rarely the fastest on an embedded board. This discrepancy arises from different hardware constraints and the often-overlooked overhead of CPU-bound post-processing, such as Non-Maximum Suppression (NMS).

Methodology

The research centers on RefineDet, a model balancing the speed of one-stage detectors with the accuracy of two-stage systems. To adapt it for edge deployment, several modifications were implemented:

  1. Reduced RefineDet (rRefineDet): A lightweight variant with reduced channel depths in the Transfer Connection Block (TCB) and intermediate layers.
  2. Backbone Exploration: A systematic evaluation of various backbones (VGG-16, ResNet-18, ResNeXt, MobileNet, Xception) to identify the most efficient architectures for embedded hardware.
  3. NMS Optimization: Tuning NMS parameters (reducing candidate boxes from 1000 to 400 and increasing confidence thresholds) to mitigate CPU bottlenecks.
mAP vs. FPS trade-off and performance across experiments

Key Insights

The Ranking Flip

A critical finding is that model rankings change across platforms. For instance, while VGG-16 was the fastest backbone on the Titan XP, ResNet-18 took the crown on the Drive PX2. This highlights the necessity of platform-specific benchmarking rather than relying solely on desktop-based metrics.

The CPU Bottleneck

While GPU scaling is relatively predictable, CPU-bound tasks like NMS do not scale linearly. On edge devices, the time spent filtering bounding boxes can occupy the majority of the inference pipeline, becoming a "silent killer" of frame rates.

Hardware Architecture

The Jetson Xavier demonstrated superior stability and performance compared to the Drive PX2, likely due to its specialized deep learning accelerators and better-balanced CPU/GPU communication.

Results

The optimized rRefineDet320 with a ResNet-18 backbone achieved 21.6 FPS on the Drive PX2 and 22.1 FPS on the Jetson Xavier, maintaining a respectable 33.5 mAP on the COCO test-dev set. Adjusting NMS parameters allowed for massive speedups on the Drive PX2 (e.g., jumping from ~10 FPS to ~20 FPS) with only a marginal hit to detection accuracy.

Detailed experimental results for various backbones and NMS settings

Conclusion

Real-time object detection on embedded platforms requires more than just porting desktop-optimized models. By addressing architectural "heads" and post-processing overheads, significant speedups are possible with minimal accuracy trade-offs. Future work should explore quantization (FP16/INT8) and detailed power consumption metrics to further enhance edge deployment.

Summary updated

BibTeX
@inproceedings{choi2020fast,
  author = {Min-Kook Choi and Jaehyung Park and Heechul Jung and Jin-Hee Lee and Soo-Heang Eo},
  title = {{Fast and Accurate Convolutional Object Detectors for Real-time Embedded Platforms}},
  booktitle = {Workshops @ ICPR},
  year = {2020},
  doi = {10.1007/978-3-030-68799-1\_43},
}
Figure