Seeking to Collide: Online Safety-Critical Scenario Generation for Autonomous Driving with Retrieval Augmented Large Language Models

Yuewen Mei, Tong Nie, Jian Sun, Ye TianOpenReview 2025 · DOI · OpenReview

TL;DR

This paper introduces an LLM-powered framework for generating realistic, safety-critical driving scenarios for autonomous vehicles (AVs) in real-time. By combining intent inference, trajectory synthesis, and a dynamic memory bank, it effectively creates challenging situations that expose AV weaknesses, significantly reducing time-to-collision and increasing collision rates compared to existing methods.

What Problem Does This Paper Address?

Validating autonomous vehicles (AVs) is crucial, and simulation plays a key role. However, current methods for generating driving scenarios often fall short. Many approaches either overfit to common driving patterns, failing to capture rare but critical "corner cases," or operate offline, meaning they can't adapt to the AV's real-time behavior during testing. This limitation means AVs might not be adequately tested on the very situations where they are most likely to fail. The paper aims to bridge this gap by creating an online, interactive, and adaptive system for generating safety-critical scenarios that can dynamically respond to AV actions and learn from new situations.

Methods

The core of the proposed solution is a retrieval-augmented Large Language Model (LLM) framework. It breaks down scenario generation into three key stages:

  1. Behavioral-Intent Inference: An LLM-based "behavior analyzer" takes the current state of the AV and surrounding vehicles and infers the most dangerous intent of a background vehicle. This is achieved through carefully engineered prompts that guide the LLM to analyze the situation, recognize features, and assign a risk level and specific behavior type (e.g., "Aggressive Cut-in," "Emergency Braking").
  2. Feasible Trajectory Generation: Once the dangerous intent is identified, another LLM-based "code generator" is prompted to create executable code for a trajectory planner. This planner then synthesizes a realistic and threatening future trajectory for the background vehicle, designed to maximize the likelihood of a collision with the ego vehicle. A "code modifier" further refines these trajectories based on simulation feedback to ensure they are both adversarial and physically plausible.
  3. Dynamic Memorization and Retrieval: To address the LLM's tendency to forget over long contexts and to speed up online generation, a dynamic memory bank is employed. This bank stores pairs of behavioral intents and their corresponding trajectory planner code. When a new scenario arises, the system first checks if a similar intent is already in the memory. If so, it retrieves the pre-generated planner, saving computation. If the intent is novel, the LLM generates a new planner and updates the memory bank. This allows the system to adapt to changing scenarios and learn new dangerous behaviors over time.

Datasets

The experiments were conducted using the Waymo Open Motion Dataset (WOMD). This dataset contains 81 scenarios, including straight-road and intersection scenarios, each featuring an ego vehicle, a critical background vehicle, and other surrounding vehicles. The authors manually annotated the most dangerous behavior of the critical background vehicle in these scenarios to serve as ground truth for evaluating the LLM behavior analyzer.

Key Insights and Ideas

  • LLMs as Scenario Architects: The paper demonstrates that LLMs can be effectively leveraged not just for text generation, but for complex reasoning and code generation required for realistic and adversarial driving scenarios.
  • Intent-Driven Adversarial Generation: Moving beyond simple trajectory perturbations, the framework focuses on inferring the intent behind a dangerous maneuver, leading to more human-like and believable adversarial actions.
  • Online Adaptation is Key: The critical insight is that safety validation requires scenarios that evolve with the AV's behavior. The online, retrieval-augmented approach directly addresses this by allowing continuous interaction and adaptation.
  • Memory for Efficiency and Novelty: The dynamic memorization and retrieval mechanism is crucial for both real-time performance (by reusing known behaviors) and for discovering genuinely novel dangerous situations.
  • Bridging the Gap Between Text and Simulation: The LLM's ability to translate high-level behavioral intents into executable code for trajectory planners is a significant step in integrating language models with simulation environments.

Key Results

  • High Accuracy in Behavior Identification: The LLM-based behavior analyzer achieved an accuracy of 81.5% (using DeepSeek-R1) in identifying the most dangerous behavior of the critical background vehicle, outperforming a previous version (74.1%).
  • Significantly Reduced Time-to-Collision (TTC): The proposed LLM-A method (with retrieval) reduced the mean minimum TTC from 1.62s to 1.08s, indicating much more critical scenarios.
  • High Collision Rate: The framework achieved a 75% collision rate, substantially higher than baselines, demonstrating its effectiveness in generating scenarios that lead to failures.
  • Realistic but Aggressive Scenarios: The generated trajectories maintained kinematic distributions close to the raw data (low KL-divergences for speed and acceleration), suggesting realism while exhibiting more aggressive maneuvers.
  • Discovery of Novel Behaviors: The system successfully identified and generated scenarios for a novel dangerous behavior ("Blind-Side High-Speed Merge") not explicitly present in the initial library.

Limitations and Future Work

  • Deterministic Intent Inference: The authors acknowledge that the current intent inference is deterministic. They suggest extending it to leverage the inherent stochasticity of LLMs for more nuanced uncertainty modeling.
  • Memory Bank Scalability: While the memory bank is dynamic, future work could explore hierarchical intents and multi-agent coordination within the memory to handle more complex adversarial interactions.
  • Dataset Generalization: Current experiments are based on the Waymo dataset. The authors propose extending the framework's evaluation to other industry-standard simulators like CARLA to demonstrate broader applicability.

Summary updated

Abstract

Simulation-based testing is crucial for validating autonomous vehicles (AVs), yet existing scenario generation methods either overfit to common driving patterns or operate in an offline, non-interactive manner that fails to expose rare, safety-critical corner cases. In this paper, we introduce an online, retrieval-augmented large language model (LLM) framework for generating safety-critical driving scenarios. Our method first employs an LLM-based behavior analyzer to infer the most dangerous intent of the background vehicle from the observed state, then queries additional LLM agents to synthesize feasible adversarial trajectories. To mitigate catastrophic forgetting and accelerate adaptation, we augment the framework with a dynamic memorization and retrieval bank of intent-planner pairs, automatically expanding its behavioral library when novel intents arise. Evaluations using the Waymo Open Motion Dataset demonstrate that our model reduces the mean minimum time-to-collision from 1.62 to 1.08 s and incurs a 75% collision rate, substantially outperforming baselines.
BibTeX
@misc{mei2025seeking,
  author = {Yuewen Mei and Tong Nie and Jian Sun and Ye Tian},
  title = {{Seeking to Collide: Online Safety-Critical Scenario Generation for Autonomous Driving with Retrieval Augmented Large Language Models}},
  year = {2025},
  eprint = {2505.00972},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2505.00972},
  howpublished = {OpenReview},
}
Figure