Stochastic Sampling Simulation for Pedestrian Trajectory Prediction

Cyrus Anderson, Xiaoxiao Du, Ram Vasudevan, Matthew Johnson-Roberson

TL;DR

This paper introduces a novel method for generating realistic synthetic pedestrian trajectories using a stochastic sampling simulation. This synthetic data, requiring only a small amount of real-world annotation, is then used to train deep learning models for pedestrian trajectory prediction, achieving performance that surpasses models trained solely on real-world data. This work offers a cost-effective solution for training robust pedestrian prediction systems, crucial for safe autonomous navigation.

What Problem Does This Paper Address?

Autonomous vehicles (AVs) need to navigate complex urban environments safely, which critically depends on their ability to accurately predict the future movements of pedestrians. Pedestrian behavior is often influenced by social interactions, making prediction challenging. While deep neural networks (DNNs) have shown great promise in this area, they typically require vast amounts of meticulously annotated real-world data. Collecting and annotating such datasets is extremely time-consuming and expensive, posing a significant bottleneck for developing and deploying these advanced prediction systems. This paper aims to alleviate this data scarcity problem by proposing a method to generate high-quality synthetic data for training these DNNs.

Methods

The core of this paper is a stochastic sampling-based simulation designed to generate realistic pedestrian trajectories. The process begins by extracting key statistics from a small set of real-world pedestrian data: the average number of pedestrians per frame and their typical walking speeds, along with their variations.

The simulation then proceeds in several steps:

  1. Sampling the number of pedestrians: The number of pedestrians in a simulated frame is sampled from a normal distribution derived from the real data statistics.
  2. Sampling walking speeds: Each simulated pedestrian is assigned a walking speed sampled from a distribution that considers the average speeds and their variations observed in the real data.
  3. Sampling trajectories: For each simulated pedestrian, a real trajectory is first sampled from the available real data. This sampled trajectory is then subjected to three types of perturbations:
  4. Translation: A random horizontal displacement is applied.
  5. Reversal: The start and end points, along with intermediate waypoints, are flipped.
  6. Truncation: The trajectory is randomly shortened. These perturbed trajectories are then smoothed using splines.

The generated synthetic trajectories are automatically annotated and used to train a state-of-the-art pedestrian trajectory prediction network, specifically Social GAN.

Datasets

  • Training Data: A small amount of real-world pedestrian trajectory data is used to extract statistical properties.
  • Evaluation Datasets:
    • ETH dataset: Contains two scenes (ETH and Hotel).
  • UCY dataset: Contains two scenes (Zara and University).

The paper uses leave-one-out cross-validation, training on data from three scenes and testing on the remaining one.

Key Insights and Ideas

  • Simulation as a Data Generator: The paper convincingly demonstrates that realistic synthetic data, generated through a well-designed simulation, can be a powerful substitute for large, manually annotated real-world datasets in training complex DNNs for trajectory prediction.
  • The Power of Stochasticity: The use of stochastic sampling for pedestrian counts, speeds, and trajectory perturbations is crucial. It introduces variability that mimics real-world scenarios, leading to more robust and generalizable learned models.
  • Social GAN Benefits from Synthetic Data: The Social GAN, known for its ability to model social interactions, shows significant performance improvements when trained on the synthetically generated data, even when the synthetic dataset is smaller than the real dataset used for training. This highlights the quality and representativeness of the generated data.
  • Statistical Properties are Key: Extracting and utilizing statistical properties (mean and variance of pedestrian counts and speeds) from even a small real dataset is sufficient to guide the simulation towards generating realistic trajectories.

Key Results

  • Improved Prediction Performance: The Social GAN trained on the "Synth-Large" synthetic dataset consistently outperformed the model trained on 100% real data across all evaluation metrics (ADE, MDE, FDE) on the ETH and UCY datasets. For instance, the average ADE for "Synth-Large" was 0.47 m, compared to 0.51 m for the 100% real data.
  • Effectiveness with Limited Real Data: Even when the synthetic dataset was generated using only 20% of the real data, the performance remained competitive and often superior to training on 100% real data.
  • Quantile-Based Performance: A qualitative analysis using quantile-based distance metrics showed that the synthetic models generally achieved lower distances to the ground truth across most percentiles, indicating more accurate predictions.
  • Ablation Study Insights: Removing pedestrian statistics (variance terms) from the sampler led to a slight increase in MDE, suggesting that capturing these variations is important for nuanced predictions.

Limitations and Future Work

  • Scene Geometry Ignored: The current Social GAN architecture and the proposed simulation method do not explicitly incorporate scene geometry. The authors suggest extending the sampling method to include scene geometry and training DNNs that leverage this information (e.g., Scene-LSTM).
  • Interaction Modeling: While the simulation generates trajectories, it doesn't explicitly model complex social interactions like pedestrian yielding. The authors propose sampling interaction outcomes as a future direction.
  • Real Data's Role in Uncertainty: While synthetic data improves overall accuracy, the paper notes that adding real data can promote the expression of uncertainty in the DNN, which is important for safety-critical applications. Future work could explore how to better imbue synthetic data with this uncertainty.

Summary updated

Abstract

Urban environments pose a significant challenge for autonomous vehicles (AVs) as they must safely navigate while in close proximity to many pedestrians. It is crucial for the AV to correctly understand and predict the future trajectories of pedestrians to avoid collision and plan a safe path. Deep neural networks (DNNs) have shown promising results in accurately predicting pedestrian trajectories, relying on large amounts of annotated real-world data to learn pedestrian behavior. However, collecting and annotating these large real-world pedestrian datasets is costly in both time and labor. This paper describes a novel method using a stochastic sampling-based simulation to train DNNs for pedestrian trajectory prediction with social interaction. Our novel simulation method can generate vast amounts of automatically-annotated, realistic, and naturalistic synthetic pedestrian trajectories based on small amounts of real annotation. We then use such synthetic trajectories to train an off-the-shelf state-of-the-art deep learning approach Social GAN (Generative Adversarial Network) to perform pedestrian trajectory prediction. Our proposed architecture, trained only using synthetic trajectories, achieves better prediction results compared to those trained on human-annotated real-world data using the same network. Our work demonstrates the effectiveness and potential of using simulation as a substitution for human annotation efforts to train high-performing prediction algorithms such as the DNNs.
BibTeX
@inproceedings{anderson2019stochastic,
  author = {Cyrus Anderson and Xiaoxiao Du and Ram Vasudevan and Matthew Johnson-Roberson},
  title = {{Stochastic Sampling Simulation for Pedestrian Trajectory Prediction}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2019},
  url = {https://openreview.net/forum?id=b3dUCxgPRJdA},
}
Figure