Abstract
This paper introduces VRU Pose-SSD, a streamlined, multitask architecture designed for joint person detection and 2D pose estimation in real-time automated driving (AD) environments. By optimizing a top-down approach using a Single Shot Detector (SSD) and TensorRT quantization, the researchers achieved a performance of 52 FPS on Full HD images. This model is up to 4.1x faster than current state-of-the-art architectures like Mask R-CNN while remaining competitive in accuracy on automotive-grade benchmarks.
Introduction
For automated driving systems, simple object detection is insufficient for safe navigation. To predict intentions—such as whether a pedestrian is about to cross or a cyclist is signaling a turn—the system requires pose estimation. However, high-accuracy models are often too computationally expensive for the strict latency and memory constraints of vehicle hardware. Furthermore, standard benchmarks like COCO do not adequately represent road-specific challenges like varying scales and frequent occlusions. VRU Pose-SSD aims to bridge this gap by providing high-fidelity pose estimation at automotive-grade speeds.
Methodology
VRU Pose-SSD utilizes a "top-down" architecture, detecting individuals first and then estimating poses within those bounding boxes. This approach is generally more efficient for real-time applications than bottom-up methods that group detected body parts.
Architecture Components
- Core Network: A modified Inception V1 backbone with two additional inception modules to expand the receptive field for objects close to the vehicle.
- Detection Head: A Single Shot Detector (SSD) replaces the slower two-stage proposal process, merging region proposal and classification.
- Pose Head: Optimized for the 17-joint COCO convention, with reduced channel counts (from 128 to 64) to boost speed without sacrificing accuracy.
- Optimization: The model employs INT8 quantization via NVIDIA TensorRT for high-speed inference.
Experimental Setup
The model was evaluated using the Tsinghua-Daimler Urban Pose Dataset (TDUP), which contains 21.4k images and over 93k instances of pedestrians and riders. Unlike general datasets, TDUP features real-world urban traffic scenes with specific annotations for occlusion and truncation, categorized into "Reasonable," "Occluded," and "Small" VRUs.
To better reflect AD safety requirements, the authors utilized Log-Average Miss Rate (LAMR) as the primary metric. LAMR penalizes false negatives (misses) more heavily than standard precision/recall, which is critical for avoiding collisions.
Key Results and Discussion
VRU Pose-SSD demonstrates a significant leap in efficiency. The optimized model runs at 52 FPS (approximately 19.41 ms per frame) on Full HD images, making it 1.6x to 4.1x faster than Mask R-CNN and Pifpaf.
Key Insights
- Quantization Efficiency: Switching from FP32 to INT8 quantization resulted in a 2.5x speedup with negligible accuracy loss.
- Rider Performance: The model performed better on riders (cyclists/motorcyclists) than pedestrians, likely due to riders appearing in clearer street contexts with less background clutter.
- Pruning: The authors found that core network capacity could be pruned by up to 50% with only marginal accuracy drops, suggesting that many SOTA models are over-parameterized for specific VRU tasks.
Conclusion and Future Work
While VRU Pose-SSD is highly effective, it still faces challenges in extremely crowded scenes where the detector may fail to distinguish heavily overlapped individuals. Common errors include "limb-swapping" (inversion) in the lower body due to fewer visual cues. Future research will focus on incorporating temporal consistency (tracking across frames) to reduce jitter and using these stable pose estimates for higher-level tasks like explicit gesture recognition and intention estimation.
Summary updated