VRU Pose-SSD: Multiperson Pose Estimation For Automated Driving

Chandan Kumar, Jayanth Ramesh, Bodhisattwa Chakraborty, Renjith Raman, Christoph Weinrich, Anurag Mundhada, Arjun Jain, Fabian B. FlohrAAAI 2021 · DOI

Abstract

This paper introduces VRU Pose-SSD, a streamlined, multitask architecture designed for joint person detection and 2D pose estimation in real-time automated driving (AD) environments. By optimizing a top-down approach using a Single Shot Detector (SSD) and TensorRT quantization, the researchers achieved a performance of 52 FPS on Full HD images. This model is up to 4.1x faster than current state-of-the-art architectures like Mask R-CNN while remaining competitive in accuracy on automotive-grade benchmarks.

Introduction

For automated driving systems, simple object detection is insufficient for safe navigation. To predict intentions—such as whether a pedestrian is about to cross or a cyclist is signaling a turn—the system requires pose estimation. However, high-accuracy models are often too computationally expensive for the strict latency and memory constraints of vehicle hardware. Furthermore, standard benchmarks like COCO do not adequately represent road-specific challenges like varying scales and frequent occlusions. VRU Pose-SSD aims to bridge this gap by providing high-fidelity pose estimation at automotive-grade speeds.

Methodology

VRU Pose-SSD utilizes a "top-down" architecture, detecting individuals first and then estimating poses within those bounding boxes. This approach is generally more efficient for real-time applications than bottom-up methods that group detected body parts.

Architecture Components

  • Core Network: A modified Inception V1 backbone with two additional inception modules to expand the receptive field for objects close to the vehicle.
  • Detection Head: A Single Shot Detector (SSD) replaces the slower two-stage proposal process, merging region proposal and classification.
  • Pose Head: Optimized for the 17-joint COCO convention, with reduced channel counts (from 128 to 64) to boost speed without sacrificing accuracy.
  • Optimization: The model employs INT8 quantization via NVIDIA TensorRT for high-speed inference.
Figure 4: The VRU Pose-SSD architecture, featuring an Inception v1 backbone, SSD detection head, and an optimized Pose-Head for joint-point heat maps.

Experimental Setup

The model was evaluated using the Tsinghua-Daimler Urban Pose Dataset (TDUP), which contains 21.4k images and over 93k instances of pedestrians and riders. Unlike general datasets, TDUP features real-world urban traffic scenes with specific annotations for occlusion and truncation, categorized into "Reasonable," "Occluded," and "Small" VRUs.

To better reflect AD safety requirements, the authors utilized Log-Average Miss Rate (LAMR) as the primary metric. LAMR penalizes false negatives (misses) more heavily than standard precision/recall, which is critical for avoiding collisions.

Key Results and Discussion

VRU Pose-SSD demonstrates a significant leap in efficiency. The optimized model runs at 52 FPS (approximately 19.41 ms per frame) on Full HD images, making it 1.6x to 4.1x faster than Mask R-CNN and Pifpaf.

Figure 5: Speed vs. Accuracy trade-off. VRU Pose-SSD (points A-J) shows significantly lower latency (Timing in ms) compared to Mask R-CNN while maintaining competitive Pedestrian LAMR.

Key Insights

  • Quantization Efficiency: Switching from FP32 to INT8 quantization resulted in a 2.5x speedup with negligible accuracy loss.
  • Rider Performance: The model performed better on riders (cyclists/motorcyclists) than pedestrians, likely due to riders appearing in clearer street contexts with less background clutter.
  • Pruning: The authors found that core network capacity could be pruned by up to 50% with only marginal accuracy drops, suggesting that many SOTA models are over-parameterized for specific VRU tasks.

Conclusion and Future Work

While VRU Pose-SSD is highly effective, it still faces challenges in extremely crowded scenes where the detector may fail to distinguish heavily overlapped individuals. Common errors include "limb-swapping" (inversion) in the lower body due to fewer visual cues. Future research will focus on incorporating temporal consistency (tracking across frames) to reduce jitter and using these stable pose estimates for higher-level tasks like explicit gesture recognition and intention estimation.

Summary updated

BibTeX
@inproceedings{kumar2021vru,
  author = {Chandan Kumar and Jayanth Ramesh and Bodhisattwa Chakraborty and Renjith Raman and Christoph Weinrich and Anurag Mundhada and Arjun Jain and Fabian B. Flohr},
  title = {{VRU Pose-SSD: Multiperson Pose Estimation For Automated Driving}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2021},
  doi = {10.1609/aaai.v35i17.17800},
}
Figure