A Work Based on GAN

Honghao FuarXiv 2023 · DOI · arXiv

Abstract

Understanding pedestrian intentions is vital for autonomous vehicle safety. However, real-world occlusions frequently result in incomplete pose data. This paper introduces SDR-GAIN (Separation and Dimensionality Reduction-based Generative Adversarial Imputation Nets), a framework that treats pose completion as a numerical coordinate imputation task. By separating body parts and reducing data dimensionality, SDR-GAIN achieves a 47.4% reduction in error compared to baselines while maintaining microsecond-level inference speeds.

Introduction

In autonomous driving, human pose estimation provides the "skeleton" data necessary to predict whether a pedestrian is walking, standing, or about to cross a street. Unfortunately, pedestrians are often partially hidden by cars, trees, or buildings. Existing vision-based solutions face a trade-off between accuracy and real-time performance. There is a critical need for a method that can accurately reconstruct complex occlusion patterns fast enough to run on a vehicle's onboard computer without significant latency.

Methods: The SDR-GAIN Framework

The authors propose shifting the focus from raw visual pixels to numerical coordinate distributions. The SDR-GAIN pipeline consists of several key stages:

  1. Initial Estimation: A standard pose estimator (e.g., OpenPose) extracts available keypoints.
  2. SDR (Separation and Dimensionality Reduction): This is the core innovation. The 18 keypoints are split into "Head" and "Body" (Torso) groups to reduce learning complexity. These coordinates are then rotated to a standard angle and projected into 1D distributions before being normalized to a [0, 1] range.
  3. Adversarial Learning: A GAN architecture is employed where a Generator fills in missing keypoints using residual modules, while a Discriminator distinguishes between real and imputed poses. A "hint" mechanism helps the discriminator focus on the specific missing parts, accelerating training.
The overall SDR-GAIN architecture, showing the flow from incomplete pose to integrated complete pose.

Key Insights and Design Choices

Several strategic decisions underpin the success of SDR-GAIN:

  • Numerical Imputation over Computer Vision: By treating pose completion as a math problem rather than a pixel-level task, the model achieves massive speedups.
  • Divide and Conquer: Separating the head and body allows the generators to specialize in the distinct movement patterns of these body parts.
  • Standardization via Rotation: Rotating coordinates to a consistent reference frame allows the GAN to learn the underlying structure of a human pose rather than memorizing specific spatial locations.
Illustration of the separation and rotation process for head and body keypoints.

Datasets and Evaluation

The model was evaluated on two major benchmarks:

  • COCO: A large-scale dataset with over 100,000 human body annotations.
  • JAAD (Joint Attention in Autonomous Driving): A dataset focused on real-world pedestrian behavior in traffic.

Results

SDR-GAIN demonstrates significant improvements over traditional methods:

  • Accuracy: It achieved at least a 47.4% reduction in RMSE compared to k-NN, MissForest, and Transformer-based interpolation.
  • Efficiency: The system achieves microsecond-level inference (~0.4 ms per pose), adding less than 2% overhead to existing detection pipelines.
  • Qualitative Performance: Visual results confirm successful reconstruction of limbs even under heavy occlusion.

Limitations and Future Work

While effective, the model's performance is dependent on the quality of training data. Future work will focus on stabilizing GAN training further and exploring larger, domain-specific datasets to enhance robustness in diverse traffic environments.

Summary updated

BibTeX
@misc{fu2023work,
  author = {Honghao Fu},
  title = {{A Work Based on GAN}},
  year = {2023},
  eprint = {2306.03538},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2306.03538},
}
Figure