Abstract
Optimizing state-of-the-art object detectors for high-end GPUs often leads to sub-optimal performance on edge devices. This study investigates the performance gap between desktop GPUs and embedded platforms like the NVIDIA Jetson Xavier and Drive PX2. By redesigning the RefineDet architecture and optimizing post-processing bottlenecks, the authors achieve real-time performance on embedded hardware without significant accuracy loss.
Introduction
Computer vision benchmarks are typically dominated by models optimized for massive desktop GPUs. However, deploying these models on edge devices reveals a disconnect: the fastest model on a Titan XP is rarely the fastest on an embedded board. This discrepancy arises from different hardware constraints and the often-overlooked overhead of CPU-bound post-processing, such as Non-Maximum Suppression (NMS).
Methodology
The research centers on RefineDet, a model balancing the speed of one-stage detectors with the accuracy of two-stage systems. To adapt it for edge deployment, several modifications were implemented:
- Reduced RefineDet (rRefineDet): A lightweight variant with reduced channel depths in the Transfer Connection Block (TCB) and intermediate layers.
- Backbone Exploration: A systematic evaluation of various backbones (VGG-16, ResNet-18, ResNeXt, MobileNet, Xception) to identify the most efficient architectures for embedded hardware.
- NMS Optimization: Tuning NMS parameters (reducing candidate boxes from 1000 to 400 and increasing confidence thresholds) to mitigate CPU bottlenecks.
Key Insights
The Ranking Flip
A critical finding is that model rankings change across platforms. For instance, while VGG-16 was the fastest backbone on the Titan XP, ResNet-18 took the crown on the Drive PX2. This highlights the necessity of platform-specific benchmarking rather than relying solely on desktop-based metrics.
The CPU Bottleneck
While GPU scaling is relatively predictable, CPU-bound tasks like NMS do not scale linearly. On edge devices, the time spent filtering bounding boxes can occupy the majority of the inference pipeline, becoming a "silent killer" of frame rates.
Hardware Architecture
The Jetson Xavier demonstrated superior stability and performance compared to the Drive PX2, likely due to its specialized deep learning accelerators and better-balanced CPU/GPU communication.
Results
The optimized rRefineDet320 with a ResNet-18 backbone achieved 21.6 FPS on the Drive PX2 and 22.1 FPS on the Jetson Xavier, maintaining a respectable 33.5 mAP on the COCO test-dev set. Adjusting NMS parameters allowed for massive speedups on the Drive PX2 (e.g., jumping from ~10 FPS to ~20 FPS) with only a marginal hit to detection accuracy.
Conclusion
Real-time object detection on embedded platforms requires more than just porting desktop-optimized models. By addressing architectural "heads" and post-processing overheads, significant speedups are possible with minimal accuracy trade-offs. Future work should explore quantization (FP16/INT8) and detailed power consumption metrics to further enhance edge deployment.
Summary updated