Abstract
With the advancement of vision-based autonomous driving technology, pedestrian detection has become an important component for improving traffic safety and driving system robustness. Nevertheless, in complex traffic scenarios, conventional pose estimation approaches frequently fail to accurately reconstruct occluded keypoints, primarily due to obstructions caused by vehicles, vegetation, or architectural elements. To address this issue, we propose a novel real-time occluded pedestrian pose completion framework termed Separation and Dimensionality Reduction-based Generative Adversarial Imputation Nets (SDR-GAIN). Unlike previous approaches that train visual models to distinguish occlusion patterns, SDR-GAIN aims to learn human pose directly from the numerical distribution of keypoint coordinates and interpolate missing positions. It employs a self-supervised adversarial learning paradigm to train lightweight generators with residual structures for the imputation of missing pose keypoints. Additionally, it integrates multiple pose standardization techniques to alleviate the difficulty of the learning process. Experiments conducted on the COCO and JAAD datasets demonstrate that SDR-GAIN surpasses conventional machine learning and Transformer-based missing data interpolation algorithms in accurately recovering occluded pedestrian keypoints, while simultaneously achieving microsecond-level real-time inference.
Introduction
In the domain of autonomous driving, understanding pedestrian intent is vital for safe navigation. Human Pose Estimation (HPE) facilitates this by identifying "keypoints" (joints like knees, elbows, and ankles). However, real-world traffic scenarios are often cluttered; pedestrians are frequently occluded by parked cars, vegetation, or other infrastructure. When a vision system cannot detect a pedestrian's limbs, the pose estimation fails, leaving the autonomous system "blind" to the person's actual stance or movement direction.
While many deep learning models attempt to solve this by increasing visual processing power, these vision-based approaches are often computationally expensive and too slow for high-speed driving environments. SDR-GAIN addresses this by treating pose completion as a numerical data imputation problem rather than a heavy computer vision task, allowing for high-speed processing without sacrificing accuracy.
Methodology
SDR-GAIN (Separation and Dimensionality Reduction-based Generative Adversarial Imputation Nets) takes the numerical coordinates of keypoints found by a standard detector and predicts missing ones using a Generative Adversarial Network (GAN). The framework utilizes a specialized "SDR" pipeline to preprocess data before imputation.
Preprocessing and Standardization
To simplify the learning task for the neural network, SDR-GAIN employs three primary preprocessing steps:
- Separation: The human body is split into two groups—head and torso. Because these regions have statistically different movement patterns, the framework uses separate, specialized generators for each.
- Rotation: To handle pedestrians at various angles, the system rotates coordinates to a consistent "standard" orientation based on reference points like ears or shoulders.
- Dimensionality Reduction: 2D coordinates are projected into 1D distributions and normalized, significantly reducing the complexity of the input data.
Adversarial Imputation
The core engine is a GAN where a "Generator" with residual structures tries to fill in missing keypoints, while a "Discriminator" attempts to distinguish real keypoints from those generated. By working with a small set of (x, y) coordinates instead of millions of pixels, the model remains incredibly lightweight. The use of Huber Loss further ensures robustness against outliers during training.
Key Results
The model was evaluated on the COCO and JAAD datasets, yielding several significant insights:
- Accuracy: SDR-GAIN achieved a 47.4% reduction in Root Mean Square Error (RMSE) compared to the original GAIN framework and other baselines like k-NN or MissForest.
- Speed: The system achieves microsecond-level inference. This is orders of magnitude faster than high-precision Transformer-based models, which is critical for vehicles traveling at high speeds.
- Qualitative Success: In real-world tests on the JAAD dataset, the model successfully reconstructed limbs hidden by car hoods where other models produced anatomically impossible "broken" poses.
Conclusion
SDR-GAIN demonstrates that treating pose completion as a coordinate imputation problem is a highly effective strategy for real-time autonomous driving applications. By combining geometric standardization with adversarial learning, the framework provides a robust solution to the problem of pedestrian occlusion. Future work will explore more advanced regularization techniques to stabilize GAN training and further integrate the system with initial pose estimators to reduce total pipeline latency.
Summary updated