Runtime Safety Assurance for Learning-enabled Control of Autonomous Driving Vehicles

Shengduo Chen, Yaowei Sun, Dachuan Li, Qiang Wang, Qi Hao, Joseph Sifakis

TL;DR

This paper introduces Simplex-Drive, a novel framework that provides runtime safety guarantees for autonomous vehicles (AVs) controlled by machine learning (ML) models. It combines a high-performance but unverified deep reinforcement learning (DRL) controller with a provably safe baseline controller, managed by a verified mode switching unit, ensuring safety without sacrificing performance.

What Problem Does This Paper Address?

The increasing reliance on ML, particularly Deep Reinforcement Learning (DRL), for controlling autonomous vehicles (AVs) presents a significant challenge: ensuring safety. DRL models, while capable of achieving impressive performance in complex scenarios, are often "black boxes" with opaque decision-making processes. This lack of transparency makes traditional formal verification methods difficult to apply during the design phase. Consequently, guaranteeing that these ML-based controllers will operate safely at runtime, especially in critical situations, remains a major hurdle for widespread AV adoption. The paper aims to bridge this gap by providing a mechanism for runtime safety assurance, even when the primary controller is unverified.

Methods

The core of the Simplex-Drive framework is its three-component architecture:

  • Advanced Controller (AC): This is an unverified DRL-based controller designed to achieve high performance in complex driving scenarios, such as lane changes. The paper details a specific DRL architecture using multi-head attention networks to map vehicle states to continuous control actions (throttle and yaw).
  • Baseline Safe Controller (BC): This is a verified controller that provides provable safety guarantees. The authors adapt the Optimal Reciprocal Collision Avoidance (ORCA) algorithm for this role, extending it to handle the kinematic constraints of Ackermann steering vehicles. ORCA ensures collision avoidance by calculating velocity obstacles and determining safe velocity sets.
  • Mode Management Unit (MM): This is a verified component responsible for monitoring the AV's state and switching control authority between the AC and BC. The MM uses a formal definition of a "recoverable region" to determine when to switch to the BC to prevent unsafe states and when it's safe to return control to the AC to maintain performance. The paper provides a formal correctness proof for the overall Simplex-Drive framework.

Datasets

Not discussed in the paper. The paper focuses on a simulated environment for training and evaluation.

Key Insights and Ideas

  • Runtime Assurance is Key: The paper highlights that for complex, unverified controllers like DRL, runtime assurance is a more practical approach than trying to achieve formal verification at design time.
  • Hybrid Control for Safety: Combining a high-performance, unverified controller with a provably safe, albeit potentially less performant, controller is an effective strategy for achieving both safety and efficiency.
  • Formal Guarantees for Switching: The formal proof of the Simplex-Drive framework, particularly the logic for switching between controllers and the definition of the "recoverable region," is crucial for establishing trust in the system.
  • Adaptability of ORCA: Extending ORCA to Ackermann steering vehicles demonstrates its flexibility and applicability to real-world automotive control problems.

Key Results

The Simplex-Drive framework demonstrated strong performance in simulated dense traffic lane-changing scenarios:

  • Zero Collisions: Across all tested densities (1, 1.5, and 2 vehicles/lane), Simplex-Drive achieved a 0% collision rate, matching the performance of the standalone ORCA-Drive controller.
  • Balanced Performance: Compared to other methods, Simplex-Drive achieved a good balance between safety (low collision rate) and efficiency (target lane rate, average speed). For instance, at a density of 1.5 vehicles/lane, it achieved a 89% target lane rate with 0 collisions, outperforming AVO and Attention PPO in terms of safety.
  • Effective Switching: Experiments simulating AC failure showed that the BC's active time increased proportionally to the aggressiveness of the "dummy" controller and traffic density, indicating the switching logic effectively intervenes when needed without excessive switching.
  • Handling Unexpected Scenarios: The framework successfully handled an example scenario where crashed vehicles blocked the lane, by identifying the unsafe state and switching to the ORCA-Drive safe control mode.

Limitations and Future Work

  • Simulation-Based: The current evaluation is based on a highway-env simulator. The authors acknowledge that real-world validation is the next crucial step.
  • Generalizability of DRL: While the framework is designed to be generalizable to various data-driven controllers, the specific DRL controller used is tailored for lane-changing.
  • Future Work: The authors plan to focus on implementing and validating the Simplex-Drive approach on physical vehicle platforms in real-world scenarios. They also suggest the framework can be extended to other data-driven or model-based controllers, aiming for a more general paradigm for trustworthy AV systems.

Summary updated

Abstract

Providing safety guarantees for Autonomous Vehicle (AV) systems with machine-learning based controllers remains a challenging issue. In this work, we propose Simplex-Drive, a framework that can achieve runtime safety assurance for machine-learning enabled controllers of AVs. The proposed Simplex-Drive consists of an unverified Deep Reinforcement Learning (DRL)-based advanced controller (AC) that achieves desirable performance in complex scenarios, a Velocity-Obstacle (VO) based baseline safe controller (BC) with provably safety guarantees, and a verified mode management unit that monitors the operation status and switches the control authority between AC and BC based on safety-related conditions. We provide a formal correctness proof of Simplex-Drive and conduct a lane-changing case study in dense traffic scenarios. The simulation experiment results demonstrate that Simplex-Drive can always ensure the operation safety without sacrificing control performance, even if the DRL policy may lead to deviations from the safe status.
BibTeX
@inproceedings{chen2022runtime,
  author = {Shengduo Chen and Yaowei Sun and Dachuan Li and Qiang Wang and Qi Hao and Joseph Sifakis},
  title = {{Runtime Safety Assurance for Learning-enabled Control of Autonomous Driving Vehicles}},
  booktitle = {IEEE International Conference on Robotics and Automation},
  year = {2022},
  doi = {10.1109/icra46639.2022.9812177},
}
Figure