AVFI: Fault Injection for Autonomous Vehicles

Saurabh Jha, Subho S. Banerjee, James Cyriac, Zbigniew T. Kalbarczyk, Ravishankar K. Iyer

TL;DR

This paper introduces AVFI, a novel fault injection framework for autonomous vehicles (AVs) that goes beyond component-level testing to assess end-to-end resilience. By simulating realistic faults in sensors, neural networks, and hardware, AVFI empirically validates AV robustness and quantifies critical failure metrics like traffic violations, highlighting the urgent need for comprehensive testing methodologies.

What Problem Does This Paper Address?

As autonomous vehicles (AVs) become more prevalent, ensuring their safety and reliability is paramount. While individual AV components are improving, their overall resilience—their ability to withstand unexpected failures and continue operating safely—is not yet at human levels. Existing research often focuses on the resilience of isolated AV systems, like perception neural networks. However, there's a significant gap in holistic, end-to-end resilience assessment methodologies that can evaluate how the entire AV system behaves under various fault conditions in realistic driving scenarios. This paper aims to bridge that gap by providing a tool and framework for such comprehensive testing.

Methods

The core of this work is AVFI (Autonomous Vehicle Fault Injector), a framework designed for end-to-end resilience assessment of AVs. AVFI integrates with a state-of-the-art AV simulation environment, CARLA, which provides realistic urban driving scenarios, diverse sensors (camera, LIDAR, GPS), and vehicle dynamics.

AVFI's methodology involves:

  • Fault Injection: It can inject faults into various parts of the AV system:
  • Sensor Inputs: Simulating faults like camera occlusions, noise, or water droplets.
  • Neural Networks: Introducing noise to parameters (weights) of the neural networks responsible for perception, localization, and planning, mimicking hardware failures or adversarial attacks.
  • Hardware/Software Components: Simulating transient or permanent faults in processing units.
  • Timing: Introducing delays or data loss in communication paths between AV components.
  • Autonomous Driving Agent (ADA): The paper utilizes an existing ADA based on an imitation learning-based Convolutional Neural Network (IL-CNN) for perception, planning, and localization.
  • Resilience Metrics: AVFI quantifies resilience using domain-specific metrics:
  • Mission Success Rate (MSR): Percentage of navigation missions successfully completed within a time limit.
  • Traffic Violations Per KM (VPK): Number of traffic violations (lane violations, driving on curbs, collisions) per kilometer driven.
  • Accidents Per KM (APK): Number of collisions with pedestrians or other vehicles per kilometer.
  • Time to Traffic Violation (TTV): Time elapsed between a fault injection and the manifestation of a traffic violation.

Datasets

The paper does not explicitly mention training datasets. The evaluation is performed using the CARLA simulator, which provides a simulated urban environment. The authors refer to real-world data from Nissan in their discussion of findings, but this is not used as an evaluation dataset within the paper itself.

Key Insights and Ideas

  • End-to-End Resilience is Crucial: The paper strongly advocates for moving beyond component-level testing to assess the overall resilience of the entire AV system.
  • Fault Injection Reveals Hidden Vulnerabilities: Introducing faults, even seemingly minor ones like sensor noise or timing delays, can significantly impact an AV's ability to navigate safely and lead to traffic violations.
  • Timing is Critical for AVs: Even small delays (e.g., 2 seconds) between an AV's decision and its physical actuation can drastically increase traffic violations, underscoring the importance of real-time performance.
  • Variability in Performance: Sensor faults, even when injected in a similar manner, can lead to a wide range of outcomes in mission success and traffic violations, highlighting the complex interactions within AV systems.
  • Need for Robust Failure Models: The findings reinforce the necessity of developing and analyzing comprehensive failure models for deep learning-based ADAs.

Key Results

  • Increased Traffic Violations: Preliminary results show that AVFI can successfully inject faults that lead to traffic violations, validating its effectiveness.
  • Variability in Mission Success: Sensor faults lead to an increase in the variability of mission success rates across different test scenarios.
  • Correlation between Success and Violations: The variability in mission success rate is correlated with an increase in traffic violations per kilometer.
  • Significant Impact of Timing Delays: Introducing delays between the AV's neural network output and its actuation resulted in a significant increase in traffic violations per kilometer. A delay of 30 frames (approximately 2 seconds at 15 FPS) led to a noticeable rise in VPK.

Limitations and Future Work

  • Limitations:
    • The paper focuses on a specific IL-CNN-based ADA and the CARLA simulator. The generalizability of findings to other AV architectures and real-world driving conditions may require further investigation.
  • The fault models used are simplified representations of real-world failures.
  • Future Work:
    • The authors suggest further experimentation and analysis of failure models and modes for deep-learning-based ADAs.
  • Exploring more complex and realistic fault injection scenarios.
  • Extending AVFI to assess resilience in a wider range of driving environments and conditions.
  • Investigating the real-time nature and constraints associated with AV systems more deeply.

Summary updated

Abstract

Autonomous vehicle (AV) technology is rapidly becoming a reality on U.S. roads, offering the promise of improvements in traffic management, safety, and the comfort and efficiency of vehicular travel. With this increasing popularity and ubiquitous deployment, resilience has become a critical requirement for public acceptance and adoption. Recent studies into the resilience of AVs have shown that though the AV systems are improving over time, they have not reached human levels of automation. Prior work in this area has studied the safety and resilience of individual components of the AV system (e.g., testing of neural networks powering the perception function). However, methods for holistic end-to-end resilience assessment of AV systems are still non-existent.
BibTeX
@inproceedings{jha2018avfi,
  author = {Saurabh Jha and Subho S. Banerjee and James Cyriac and Zbigniew T. Kalbarczyk and Ravishankar K. Iyer},
  title = {{AVFI: Fault Injection for Autonomous Vehicles}},
  booktitle = {DSN Workshops},
  year = {2018},
  url = {https://openreview.net/forum?id=IFaVXeHMq99},
}
Figure