TrafficSim: Learning To Simulate Realistic Multi-Agent Behaviors

Simon Suo, Sebastian Regalado, Sergio Casas, Raquel Urtasun

Abstract

Simulation has the potential to massively scale evaluation of self-driving systems, enabling rapid development as well as safe deployment. Bridging the gap between simulation and the real world requires realistic multi-agent behaviors. Existing simulation environments rely on heuristic-based models that directly encode traffic rules, which cannot capture irregular maneuvers (e.g., nudging, U-turns) and complex interactions (e.g., yielding, merging). In contrast, we leverage real-world data to learn directly from human demonstration, and thus capture more naturalistic driving behaviors. To this end, we propose TrafficSim, a multi-agent behavior model for realistic traffic simulation. In particular, we parameterize the policy with an implicit latent variable model that generates socially-consistent plans for all actors in the scene jointly. To learn a robust policy amenable for long horizon simulation, we unroll the policy in training and optimize through the fully differentiable simulation across time. Our learning objective incorporates both human demonstrations as well as common sense. We show TrafficSim generates significantly more realistic traffic scenarios as compared to a diverse set of baselines. Notably, we can exploit trajectories generated by TrafficSim as effective data augmentation for training better motion planner.

Introduction

Testing self-driving vehicles (SDVs) in the real world is both expensive and inherently risky. While replaying recorded logs offers a safer alternative, these "dead" logs lack reactivity; background actors do not respond if the SDV deviates from the original recorded path. To address this, high-fidelity simulation is required. However, traditional simulators often fall into two traps: rigid heuristic models (like the Intelligent Driver Model) that fail to capture human nuances, and pure imitation learning models that suffer from compounding errors during long-horizon simulations. TrafficSim bridges this gap by learning realistic, reactive multi-agent behaviors directly from human data.

Methods

TrafficSim is designed as a multi-agent behavior model for long-horizon, reactive simulation. Its architecture is built upon three core pillars:

  1. Joint Behavior Modeling: Utilizing a Variational Auto-Encoder (VAE) framework combined with a Graph Neural Network (GNN), TrafficSim captures scene-level latent variables. This ensures that the plans for all actors are socially consistent—for instance, if one vehicle merges, the following vehicle appropriately yields.
  2. Closed-Loop Differentiable Simulation: Unlike open-loop models, TrafficSim "unrolls" the simulation over a long time horizon (e.g., 12 seconds) during training. Because the entire pipeline is differentiable, the model can backpropagate losses from the end of a sequence to the initial actions, teaching it the long-term consequences of its decisions.
  3. Multi-Task Learning Objective: The model balances an imitation loss (reconstructing human trajectories) with a "common sense" collision loss. The latter uses a differentiable relaxation of car shapes to provide a smooth gradient signal that discourages collisions.
The TrafficSim simulation pipeline, from layout to rendering.

Key Insights and Results

A critical finding of this work is the superiority of closed-loop training. By seeing its own previous predictions during training, the model learns to recover from minor errors rather than spiraling into unrealistic states. Furthermore, by making the collision loss differentiable, the authors transform a hard constraint into a useful learning signal.

TrafficSim achieves state-of-the-art realism, recording the lowest Scenario Collision Rate (0.50%) among learning-based baselines while maintaining high fidelity to human trajectories (minSFDE of 1.13m). Beyond mere replication, the model can generate diverse, plausible futures for any given scene and can be further optimized at inference time to satisfy specific constraints.

TrafficSim captures a wide range of irregular maneuvers and complex social interactions.

Perhaps most significantly, the researchers demonstrated that a motion planner trained on synthetic data generated by TrafficSim outperformed one trained on the same amount of raw real-world data. This suggests that TrafficSim effectively distills the underlying intent and social logic of human driving, making it a powerful tool for data augmentation.

Conclusion and Future Work

TrafficSim represents a significant step toward realistic, reactive traffic simulation. While it excels at agent-to-agent interactions and irregular maneuvers, future work will focus on incorporating explicit map-based constraints, such as strict traffic light compliance. Additionally, the team aims to enable controllable simulation, allowing users to specify high-level goals for specific actors while maintaining naturalistic reactions from the surrounding traffic.

Summary updated

Abstract

Simulation has the potential to massively scale evaluation of self-driving systems, enabling rapid development as well as safe deployment. Bridging the gap between simulation and the real world requires realistic multi-agent behaviors. Existing simulation environments rely on heuristic-based models that directly encode traffic rules, which cannot capture irregular maneuvers (e.g., nudging, U-turns) and complex interactions (e.g., yielding, merging). In contrast, we leverage real-world data to learn directly from human demonstration, and thus capture more naturalistic driving behaviors. To this end, we propose TrafficSim, a multi-agent behavior model for realistic traffic simulation. In particular, we parameterize the policy with an implicit latent variable model that generates socially-consistent plans for all actors in the scene jointly. To learn a robust policy amenable for long horizon simulation, we unroll the policy in training and optimize through the fully differentiable simulation across time. Our learning objective incorporates both human demonstrations as well as common sense. We show TrafficSim generates significantly more realistic traffic scenarios as compared to a diverse set of baselines. Notably, we can exploit trajectories generated by TrafficSim as effective data augmentation for training better motion planner.
BibTeX
@inproceedings{suo2021trafficsim,
  author = {Simon Suo and Sebastian Regalado and Sergio Casas and Raquel Urtasun},
  title = {{TrafficSim: Learning To Simulate Realistic Multi-Agent Behaviors}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2021},
  url = {https://openreview.net/forum?id=7GOF2Wn2_RK},
}
Figure