A Multi-Vehicle Trajectories Generator to Simulate Vehicle-to-Vehicle Encountering Scenarios

Wenhao Ding, Wenshuo Wang, Ding Zhao

TL;DR

Generating realistic multi-vehicle interaction data is crucial for testing autonomous vehicles, but real-world edge cases are scarce. This paper introduces a Multi-vehicle Trajectory Generator (MTG) that uses a bi-directional recurrent encoder and a multi-branch decoder to simulate realistic vehicle-to-vehicle encounters. It also proposes a novel metric to evaluate how well the model isolates different driving behaviors in its latent space, proving MTG outperforms baselines like InfoGAN and $\beta$-VAE.

What Problem Does This Paper Address?

To safely deploy autonomous vehicles (AVs), the underlying algorithms must be trained and tested on a massive variety of complex driving scenarios, particularly multi-vehicle interactions (encounters). However, real-world datasets lack sufficient examples of these diverse, complex encounters due to the high cost and difficulty of data collection.

While deep generative models can theoretically create synthetic data to fill this gap, existing approaches struggle with multi-agent time-series data. Standard models often fail to capture the complex spatial and temporal dynamics between interacting vehicles, resulting in physically impossible trajectories (like cars making instant 90-degree turns) or "entangled" models where you cannot easily control specific features of the generated scenario.

Methods

The authors frame the problem as a sequence-to-sequence generation task and propose the Multi-vehicle Trajectory Generator (MTG).

  • Bi-directional GRU Encoder: Instead of a standard feed-forward network, MTG uses a bi-directional Gated Recurrent Unit (GRU) to process the trajectories of multiple vehicles simultaneously. This allows the model to capture both forward and backward temporal dependencies in the driving data.
  • Multi-Branch Decoder: To generate new trajectories, the decoder is split into multiple branches (one for each vehicle). These branches share hidden states to maintain contextual awareness of the other vehicle, but generate their specific coordinates independently. This prevents the generation of one sequence from overwhelmingly interfering with the other.
  • A Novel Disentanglement Metric: "Disentanglement" means that each variable in the model's latent space controls exactly one distinct feature of the output (e.g., one variable controls approach angle, another controls speed). Previous metrics relied on training linear classifiers to measure this, which made them highly sensitive to hyperparameter tuning. The authors propose a new, classifier-free metric that measures the variance of output latent codes when specific input dimensions are perturbed, offering a more stable measure of variable independence.

Datasets

  • UMTRI Dataset: The model was trained on real-world driving encounter data collected by the University of Michigan Transportation Research Institute. It consists of GPS latitude and longitude data from approximately 3,500 vehicles, sampled at 10 Hz. The trajectories were normalized and scaled to fixed sequences of 50 steps for training.

Key Insights and Ideas

  • Multi-branching is key for multi-agent generation: Forcing a single decoder to output multiple interacting trajectories often leads to messy, unrealistic data. Separating the decoder into distinct branches that only share hidden states strikes the perfect balance between agent independence and environmental context.
  • GANs struggle with trajectory diversity: The authors tested an InfoGAN baseline, which suffered heavily from mode collapse. Because the GAN's generator only wants to fool the discriminator, it found a narrow set of "safe" trajectories and generated them repeatedly, failing to provide the diverse scenarios needed for AV testing.
  • Disentanglement equals control: By achieving a highly disentangled latent space, MTG allows engineers to purposely tweak specific aspects of a driving encounter. By sliding the value of a single latent variable, an engineer can smoothly change the location where two vehicles meet or their approach directions, creating highly targeted test scenarios.

Key Results

  • Qualitative Realism: MTG generated smooth, realistic trajectories. Unlike the $\beta$-VAE baseline, which occasionally produced sharp, physically impossible turns, MTG's outputs respected the physical constraints of real vehicles.
  • Traffic Rationality: When measured against real traffic data, MTG's generated trajectories closely matched real-world distributions for vehicle distance, speed variation, and trajectory smoothness (direction variation). InfoGAN, by contrast, showed almost zero variance due to mode collapse.
  • Superior Disentanglement: Using the newly proposed metric, MTG showed almost zero dependency among its latent codes—meaning changing one code doesn't accidentally alter other features. The $\beta$-VAE baseline, however, showed significant entanglement, with one latent code heavily influencing two others.

Limitations and Future Work

  • Limitations: The authors note that the model currently only considers relatively short trajectory sequences. Furthermore, while MTG is more robust than the baselines, the ratio of output variance to input variance in the latent space is still greater than 1, indicating the encoder does not perfectly recognize all trajectories generated by the decoder.
  • Future Work: The authors plan to incorporate road profiles and map data using conditional deep generative models. They also aim to carefully cascade these short trajectories to generate longer, more complex, and sustained traffic scenarios for training autonomous driving algorithms.

Summary updated

Abstract

Generating multi-vehicle trajectories from existing limited data can provide rich resources for autonomous vehicle development and testing. This paper introduces a multi-vehicle trajectory generator (MTG) that can encode multi-vehicle interaction scenarios (called driving encounters) into an interpretable representation from which new driving encounter scenarios are generated by sampling. The MTG consists of a bi-directional encoder and a multi-branch decoder. A new disentanglement metric is then developed for model analyses and comparisons in terms of model robustness and the independence of the latent codes. Comparison of our proposed MTG with β-VAE and InfoGAN demonstrates that the MTG has stronger capability to purposely generate rational vehicle-to-vehicle encounters through operating the disentangled latent codes. Thus the MTG could provide more data for engineers and researchers to develop testing and evaluation scenarios for autonomous vehicles.
BibTeX
@inproceedings{ding2019multi,
  author = {Wenhao Ding and Wenshuo Wang and Ding Zhao},
  title = {{A Multi-Vehicle Trajectories Generator to Simulate Vehicle-to-Vehicle Encountering Scenarios}},
  booktitle = {IEEE International Conference on Robotics and Automation},
  year = {2019},
  url = {https://openreview.net/forum?id=I1itg2_rBcm},
}

Loading related content...

Figure