InterSim: Interactive Traffic Simulation via Explicit Relation Modeling

Qiao Sun, Xin Huang, Brian C. Williams, Hang Zhao

TL;DR

This paper introduces InterSim, an interactive traffic simulator that models explicit agent interactions to generate more realistic and reactive multi-agent behaviors. By reasoning about "who yields to whom," InterSim improves simulation consistency and allows for more robust testing of autonomous driving planners, especially in complex, dense traffic scenarios.

What Problem Does This Paper Address?

Testing autonomous driving (AD) systems is crucial, but real-world road testing is expensive, time-consuming, and potentially dangerous. Traditional simulators often fall short by either replaying static logs (lacking reactivity to novel ego-agent plans) or using simplified models that don't capture complex multi-agent interactions. Existing machine learning-based simulators can generate realistic individual agent trajectories but often fail to ensure consistency and avoid collisions when multiple agents interact, especially when the ego agent's plan deviates from the training data. The core challenge is to create a simulator that can generate diverse, consistent, and reactive multi-agent behaviors that accurately reflect real-world interactions, particularly in dense traffic where subtle cues dictate behavior.

Methods

InterSim tackles this by explicitly modeling interaction relations between agents. Given an ego agent's planned trajectory, InterSim first identifies "relevant" agents whose future paths might collide. For these relevant agents, it infers an "interaction relation" (e.g., "influencer" or "reactor") using a trained relation predictor. This relation then guides a goal-driven trajectory predictor to generate a reactive trajectory for the "reactor" agent, ensuring consistency with the "influencer." This process is applied iteratively to resolve any new conflicts that arise between environment agents themselves after their trajectories are updated. This explicit relation modeling allows for more interpretable and controllable simulation compared to implicit latent variable approaches.

Datasets

The authors train and evaluate InterSim on the Waymo Open Motion Dataset (WOMD), specifically focusing on its interactive dataset. They use a simulation frequency of 2Hz and simulate agent behaviors over an 8-second horizon.

Key Insights and Ideas

  • Explicit Relations Matter: The core insight is that explicitly modeling "who yields to whom" is crucial for realistic and consistent multi-agent simulation, especially in interactive scenarios. This is more interpretable and controllable than implicit latent variable models.
  • Efficiency Through Relevance: By focusing only on "relevant" agents (those whose trajectories are affected by the ego plan), InterSim achieves better simulation efficiency than methods that might consider all agents or rely on computationally expensive joint trajectory generation.
  • Balancing Realism and Reactivity: InterSim aims for a balance between simulating realistic agent behaviors (low displacement error) and ensuring reactive behaviors that avoid collisions (low collision rate). The paper shows how different variants of their model achieve different trade-offs.
  • Manipulating Scenarios: The explicit relation modeling allows for scenario manipulation. By reversing the predicted interaction relation, InterSim can simulate rare or diverse events that might be difficult to capture with purely data-driven generative models.

Key Results

  • Simulation Realism: InterSim achieves the lowest Average Displacement Error (ADE) and Final Displacement Error (FDE) compared to baselines like SimNet, indicating higher simulation realism.
  • Simulation Reactivity: InterSim significantly reduces collision rates (front, side, and rear) compared to other methods, particularly in challenging scenarios like a slowing-down ego agent. While not zero, the collision rates are substantially lower.
  • Simulation Efficiency: InterSim only needs to modify a small percentage of environment agents (at most 26% for Task 1, 11.06% for Task 2) per step, demonstrating its efficiency.
  • Simulation Progress: InterSim shows good progress (goal achievement) while maintaining low collision rates, suggesting it doesn't overly penalize agent movement to avoid collisions.

Limitations and Future Work

  • Collision Rate Not Zero: The authors acknowledge that collision rates are not entirely eliminated. This is attributed to the inherent instability of the goal-driven trajectory prediction model with novel inputs and potential noise in perception data (bounding boxes, agent positions) leading to pre-existing collisions in the dataset.
  • Future Work: The paper suggests that future work could explore more robust conflict resolution strategies and potentially investigate how to better handle scenarios with pre-existing collisions in the input data. The ability to manipulate interaction relations also opens doors for generating more diverse and rare scenarios for testing.

Summary updated

Abstract

Interactive traffic simulation is crucial to autonomous driving systems by enabling testing for planners in a more scalable and safe way compared to real-world road testing. Existing approaches learn an agent model from large-scale driving data to simulate realistic traffic scenarios, yet it remains an open question to produce consistent and diverse multi-agent interactive behaviors in crowded scenes. In this work, we present InterSim, an interactive traffic simulator for testing autonomous driving planners. Given a test plan trajectory from the ego agent, InterSim reasons about the interaction relations between the agents in the scene and generates realistic trajectories for each environment agent that are consistent with the relations. We train and validate our model on a large-scale interactive driving dataset. Experiment results show that InterSim achieves better simulation realism and reactivity in two simulation tasks compared to a state-of-the-art learning-based traffic simulator.
BibTeX
@inproceedings{sun2022intersim,
  author = {Qiao Sun and Xin Huang and Brian C. Williams and Hang Zhao},
  title = {{InterSim: Interactive Traffic Simulation via Explicit Relation Modeling}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2022},
}
Figure