Abstract
Predicting pedestrian behavior is critical for the safety and efficiency of autonomous driving systems. This paper introduces a framework that jointly forecasts a pedestrian's future trajectory and body pose from an egocentric perspective. By disentangling global movement from local limb dynamics and utilizing a specialized pose-completion module, the proposed method achieves state-of-the-art results even when trained on noisy, machine-generated labels.
Introduction: The Egocentric Challenge
Predicting pedestrian movement is a fundamental challenge for autonomous systems. Traditional research often separates trajectory forecasting (2D map positioning) from pose forecasting (limb movement). However, in real-world driving scenarios, these elements are deeply intertwined. For instance, a pedestrian's posture often signals an imminent change in direction or speed.
Capturing this data from a moving vehicle (egocentric view) introduces significant noise due to camera motion (egomotion) and frequent occlusions. Furthermore, large-scale datasets with manual joint-level annotations are scarce, necessitating models that can learn from imperfect supervision.
Methodology: A Divide and Conquer Approach
The authors propose a three-stage pipeline to forecast "locomotion"—the combination of pose and trajectory.
1. Noisy Supervision and Pose Completion
To overcome the lack of manual labels, the framework uses off-the-shelf tools like OpenPose to generate initial keypoints. A Symmetric Autoencoder (Pose Completion module) is then employed to denoise these labels and fill in missing joints caused by occlusion.
2. Disentanglement of Dynamics
The core innovation lies in splitting motion into two distinct streams:
- Global Stream: Tracks the "root" joint (the neck) to capture the overall path. This stream incorporates external signals like monocular depth and camera egomotion.
- Local Stream: Tracks all other joints relative to the neck, capturing the kinematics of walking or arm swinging.
3. Hierarchical Forecasting
The model utilizes Quasi-Recurrent Neural Networks (QRNNs) for forecasting. QRNNs offer better parallelization and faster training speeds than standard LSTMs while effectively handling the temporal dependencies of human motion.
Experimental Results
The framework was evaluated on the JAAD (Joint Attention in Autonomous Driving) dataset, augmented with monocular depth (SuperDepth) and unsupervised egomotion estimation.
Performance Metrics
The primary metric used is Keypoint Displacement Error (KDE), measuring the average pixel distance between predicted and actual joint locations.
| Model | KDE (Lower is Better) |
|---|---|
| Structural RNN | 31.7 |
| GRU-ED | 23.8 |
| Proposed Method | 10.9 |
Key Findings
- Disentanglement is Essential: Ablation studies showed that removing the disentanglement module caused the KDE to jump from 10.9 to 18.2.
- Robustness to Noise: The pose completion module successfully "cleans" machine-generated labels, proving that high-quality human annotations are not strictly necessary for training effective models.
- Efficiency: The QRNN-based architecture trained faster (825 ms/batch) than LSTM-based alternatives (886 ms/batch) with superior convergence.
Conclusion and Future Directions
This research demonstrates that separating global pathing from local kinematics significantly simplifies the task of locomotion forecasting. While the current model relies on pre-existing bounding boxes and produces 2D outputs, it sets a strong foundation for future work in social disentanglement—modeling how multiple agents interact—and extending predictions into full 3D space.
Summary updated