ECSAS: Exploring Critical Scenarios from Action Sequence in Autonomous Driving

Shuting Kang, Heng Guo, Peng Su, Lijun Zhang, Guangzhen Liu, Yunzhi Xue, Yanjun Wu

TL;DR

This paper introduces ECSAS, a novel framework for generating critical scenarios in autonomous driving by modeling action sequences. By using a new description language (BTScenario) and reinforcement learning with specialized optimizations, ECSAS effectively explores the complex parameter space to find scenarios that challenge autonomous systems, outperforming existing methods in collision rate.

What Problem Does This Paper Address?

Autonomous driving systems (ADS) require rigorous testing to ensure safety, especially in "critical scenarios" – situations that could lead to accidents. The challenge lies in the vast, continuous space of possible scenarios. Existing methods often struggle to efficiently explore this space, either by sampling parameters randomly or by considering correlations only at the initial stage of a scenario. This paper addresses the bottleneck of modeling and exploring action sequences within a scenario, recognizing that the temporal dependencies between actions and their parameters are crucial for discovering rare but critical failure modes. The goal is to move beyond static parameter sampling to a dynamic, sequential exploration that better mimics real-world driving complexities.

Methods

The core of ECSAS is its two-module architecture:

  1. Scenario Representation (BTScenario): A new domain-specific language (DSL) is introduced to model logical scenarios. BTScenario allows for the explicit representation of action sequences using temporal operators like "serial" and "parallel." It supports defining map elements, initial actor states, atomic actions (e.g., followLane, changeLane), and their parameters with ranges, as well as pre/post-conditions and test oracles (e.g., collision detection). This sequential modeling is a key differentiator.
  2. Scenario Generation Model: This module employs Reinforcement Learning (RL) to search for critical parameter combinations. Specifically, it uses the TD3 (Twin Delayed Deep Deterministic Policy Gradient) algorithm, which is suitable for continuous action spaces.
  3. Action Masking: To manage the complexity of action sequences, an action mask is used to fix the length and order of actions within a step, guiding the RL agent.
  4. Replay Buffer Optimization: To address the scarcity of critical (colliding) scenarios, the replay buffer is split into positive (collision) and negative (non-collision) buffers. This allows for more efficient sampling of critical experiences during training.
  5. Reward Design: The RL agent is rewarded for generating colliding scenarios, incentivizing the discovery of failure cases.

The ECSAS framework integrates these components with the Carla simulator to generate and evaluate concrete critical scenarios.

Datasets

The paper does not explicitly mention training on large datasets of pre-existing scenarios. Instead, it uses the Carla simulator and a virtual town map (based on Apollo format) for generating and evaluating scenarios. The experiments involve creating specific scenario types, such as "overtaking" and "turning" scenarios, to test the effectiveness of ECSAS against other methods.

Key Insights and Ideas

  • Action Sequences Matter: The most significant insight is that modeling and exploring scenarios as sequences of actions, rather than as a single static event, is crucial for discovering critical failure modes. The temporal dependencies between actions and their parameters are key.
  • BTScenario's Expressiveness: The proposed BTScenario language offers a flexible and convenient way to represent complex action sequences, bridging the gap between abstract logical scenarios and concrete, executable test cases.
  • RL for Efficient Exploration: Reinforcement learning, particularly with the TD3 algorithm, proves effective in navigating the high-dimensional parameter space to find rare critical scenarios, especially when combined with specialized optimizations.
  • Optimizations are Crucial: The action mask and optimized replay buffer are not just minor tweaks but essential components that significantly improve the efficiency and effectiveness of the RL-based scenario generation process, particularly for rare events.

Key Results

The paper demonstrates that ECSAS significantly outperforms native approaches like Random Testing (RT) and Combination Testing (CT) in terms of collision rate (CR).

  • Overtaking Scenario: ECSAS achieved a collision rate of 76.86%, a substantial boost compared to RT (30.25%) and CT (16.53%).
  • Turning Scenario: ECSAS achieved a collision rate of 75.21%, again outperforming RT (14.88%) and CT (32.72%).
  • Ablation Studies: The results also show that the action mask (AM) and replay buffer optimization (RB) individually contribute to ECSAS's performance, with the full ECSAS model yielding the highest collision rates.
  • Action Value Analysis (Fig. 5): Visualizations show that while initial action values leading to collisions in ECSAS can be discrete, they tend to gather towards specific regions in later steps, indicating that the action sequences are becoming more efficient and focused on generating critical outcomes.

Limitations and Future Work

  • Mode Collapse: The authors acknowledge a potential issue of "mode collapse" in RL, where the generation of action values might become aggregated in a single action space, leading to a lack of diversity in critical scenarios. They plan to address this in future work.
  • RL Algorithm Exploration: The paper suggests exploring different RL algorithms to analyze perception, decision, and control algorithms within the ECSAS framework.
  • BTScenario Extension: Future work includes extending BTScenario to support more complex features like random actions, random events, and distribution functions for parameter ranges.
  • Zero Values in Later Steps: For scenarios where collisions occur early, parameters in later steps might become irrelevant (e.g., speed and distance become zero). The paper notes this observation but doesn't propose a specific solution for it.

Summary updated

Abstract

Rare critical scenarios are crucial to verify the performance of autonomous driving in different situations. Critical scenario generation requires the ability of sampling critical combinations from an infinite parameter space in the logical scenario. Existing solutions aim to explore the correlation of action parameters in the initial scenario rather than action sequences. How to model action sequences so that one can further consider the effects of different action parameters is the bottleneck of the problem. In this paper, we solve the problem by proposing the ECSAS framework. Specifically, we first propose a description language, BTScenario, allowing us to model action sequences of scenarios. We then use reinforcement learning to search for combinations of critical action parameters. Several optimizations are proposed to increase efficiency, including action mask and replay buffer. Experimental results show that our model with strong collision ability and effectively outperforms the existing methods on various nontrivial scenarios.
BibTeX
@inproceedings{kang2023ecsas,
  author = {Shuting Kang and Heng Guo and Peng Su and Lijun Zhang and Guangzhen Liu and Yunzhi Xue and Yanjun Wu},
  title = {{ECSAS: Exploring Critical Scenarios from Action Sequence in Autonomous Driving}},
  booktitle = {ATS},
  year = {2023},
  url = {https://openreview.net/forum?id=wUhU7hmvgD},
}
Figure