Abstract
Long-tail and rare event problems become crucial when autonomous driving algorithms are applied in the real world. For the purpose of evaluating systems in challenging settings, we propose a generative framework to create safety-critical scenarios for evaluating specific task algorithms. We first represent the traffic scenarios with a series of autoregressive building blocks and generate diverse scenarios by sampling from the joint distribution of these blocks. We then train the generative model as an agent (or a generator) to investigate the risky distribution parameters for a given driving algorithm being evaluated. We regard the task algorithm as an environment (or a discriminator) that returns a reward to the agent when a risky scenario is generated. Through the experiments conducted on several scenarios in the simulation, we demonstrate that the proposed framework generates safety-critical scenarios more efficiently than grid search or human design methods. Another advantage of this method is its adaptiveness to the routes and parameters.
Introduction
Testing autonomous vehicles (AVs) is a paradoxical challenge: they must handle rare, dangerous situations perfectly, yet these "long-tail" events are inherently difficult to find in real-world data. Standard training datasets often lack the high-risk edge cases necessary for robust safety evaluation. Traditional approaches, such as exhaustive grid searches or manual scenario design, are either computationally inefficient or lack scalability and diversity. This paper presents a generative framework that uses Reinforcement Learning (RL) to automatically and adaptively discover safety-critical traffic scenarios tailored to specific driving algorithms.
Methodology
Scenario Representation
The core of the proposed method is the decomposition of traffic scenes into "building blocks," such as spawn positions, orientations, and trigger distances. To capture the logical relationships between these parameters, the framework employs an autoregressive graphic model (a Bayesian Network). This structure ensures that the choice of one parameter influences the probability of subsequent ones, reflecting real-world constraints—for example, a cyclist's orientation is typically relative to their starting position on the road.
Reinforcement Learning Framework
The generation process is modeled as an RL loop where a generative model acts as the agent and the driving algorithm under test (e.g., a PID-controlled vehicle in the CARLA simulator) serves as the environment. The agent outputs scenario parameters and receives a reward based on the risk generated, such as a collision or a near-miss. To maintain realism, the reward function includes penalties for "unreasonable" scenarios, such as obstacles spawning directly on top of the AV.
The optimization is performed using the REINFORCE algorithm, enhanced with an entropy regularization term. This term is critical for maintaining diversity, preventing the generator from repeatedly producing the same "perfect" crash scenario and instead forcing it to explore the wide distribution of potential risks.
Experimental Results
The framework was evaluated using the CARLA simulator, an open-source platform for autonomous driving research. The researchers utilized the Carla Scenario Runner Library to implement and "attack" standard traffic setups, such as unprotected left turns.
The experiments yielded several key insights:
- High Collision Rates: The proposed method consistently generated scenarios with significantly higher collision rates than random sampling or human-designed baselines.
- Efficiency: By modeling dependencies between variables, the framework reached stable, high-risk generation in far fewer iterations than independent policy versions.
- Adversarial Adaptivity: The method is inherently adaptive. If a driving algorithm is updated to fix a bug, the generator automatically identifies new failure modes, ensuring continuous safety improvement.
- Generalization: Once trained, the model could generate risky scenarios for new routes and speeds not seen during training, demonstrating that it learned the underlying logic of safety-critical interactions.
Conclusion and Future Work
"Learning to Collide" offers a scalable and adaptive solution to the long-tail problem in AV testing. While currently focused on interactions between an ego vehicle and a single dynamic obstacle, future work will aim to scale the complexity to multi-actor urban environments. Additionally, exploring implicit scene representations could allow the system to discover even more creative and unexpected failure modes, further pushing the boundaries of autonomous driving safety.
Summary updated