Modular Fault Diagnosis Framework for Complex Autonomous Driving Systems

Stefan Orf, Sven Ochs, Jens Doll, Albert Schotschneider, Marc Heinrich, Marc René Zofka, J. Marius Zöllner

TL;DR

This paper introduces a modular, state-aware fault diagnosis framework designed to handle the immense complexity of autonomous driving (AD) systems. By mapping dependencies between software components and considering the vehicle's current operational state (e.g., driving vs. idling), the framework filters out "noise" from propagated errors, allowing operators to identify root causes quickly and accurately.

What Problem Does This Paper Address?

Modern autonomous driving systems are "heterogeneous"—they combine traditional rule-based algorithms with AI-driven components, all running on diverse hardware. Because these parts are highly interdependent, a single sensor failure can trigger a "waterfall" of errors across perception, localization, and planning modules.

For a safety driver or a remote operator, this results in information overload: when everything reports an error, it is nearly impossible to tell what actually broke. Furthermore, some components are only relevant in specific contexts (e.g., a motion planner isn't needed while the bus is at a standstill). Existing diagnostic tools often lack the nuance to handle these dependencies and operational states, leading to "false alarms" or confusing diagnostic reports.

Methods

The authors propose a three-pronged approach to modular diagnosis:

  1. A New Classification Scheme: They categorize faults based on Location (isolated vs. contextual), Information Type (meta-properties like frequency vs. actual data content), and Data Flow (sequential vs. parallel).
  2. Discrete Diagnostic States: Instead of a simple binary "Pass/Fail," they use a five-state model: OK, WARNING, ERROR, IGNORE, and UNKNOWN. The IGNORE state is particularly novel, used when a component’s status is irrelevant to the current vehicle state.
  3. Dependency-Aware Aggregation: The framework uses a directed graph to represent how components rely on one another. If a "root" component (like a LiDAR sensor) fails, the system automatically moves dependent modules (like Perception) into the IGNORE state. This prevents error propagation from cluttering the diagnostic interface.

The system was implemented using a generic analyzer that filters diagnostic messages via regular expressions and namespaces, making it highly scalable and customizable for different vehicle architectures.

Datasets

The paper does not use standard public datasets (like Waymo or KITTI). Instead, the framework was evaluated through real-world implementation on AD shuttle buses and controlled failure injection scenarios. The authors induced specific hardware and software faults (e.g., sensor outages, localization drift, and planning timeouts) to test how the framework categorized and reported the issues in real-time.

Key Insights and Ideas

  • The "Ignore" State is Essential: By explicitly marking dependent components as IGNORE when their inputs are already known to be faulty, the system provides a much cleaner "root cause" analysis for the operator.
  • Context Matters: A failure in the planning module is critical while driving but irrelevant while the vehicle is parked at a bus stop. The framework uses a state machine to monitor the vehicle's mission, silencing non-critical alerts based on the current task.
  • Meta-Information vs. Content: The authors highlight that you don't always need to look at complex sensor data to find a fault. Often, "meta" signals—like the frequency of a message or a simple "heartbeat" (watchdog)—are faster and more reliable indicators of a total component failure.

Key Results

The framework was validated across six distinct scenarios:

  • Scenario 1 & 2 (Sensor Outage): Without dependency analysis, a sensor failure caused an "error storm" across the whole stack. With the framework, only the Sensors group showed an ERROR, while Localization and Perception were correctly moved to IGNORE.
  • Scenario 3 (Localization Failure): The system successfully isolated a localization fault and immediately flagged it, while correctly identifying that the Sensors (the input) were still healthy.
  • Scenario 4-6 (Planning Problems): These scenarios proved state-awareness. When the vehicle was "Active" (driving), a planning timeout triggered an ERROR. When the vehicle was "Inactive" (at a stop), the same timeout was correctly reported as IGNORE.

Limitations and Future Work

The primary limitation is that the dependency graph currently requires expert knowledge to build. A developer must manually define which components depend on which others.

The authors suggest that future work should focus on automatic dependency detection. By observing data flows and correlations in real-time, the system could potentially "learn" the architecture of the AD stack and update its own diagnostic graph without human intervention. Additionally, they aim to refine the "countermeasures" (like emergency braking) to be more granular based on the specific type of fault detected.

Summary updated

Abstract

Fault diagnosis is crucial for complex autonomous mobile systems, especially for modern-day autonomous driving (AD). Different actors, numerous use cases, and complex heterogeneous components motivate a fault diagnosis of the system and overall system integrity. AD systems are composed of many heterogeneous components, each with different functionality and possibly using a different algorithm (e.g., rule-based vs. AI components). In addition, these components are subject to the vehicle's driving state and are highly dependent. This paper, therefore, faces this problem by presenting the concept of a modular fault diagnosis framework for AD systems. The concept suggests modular state monitoring and diagnosis elements, together with a state- and dependency-aware aggregation method. Our proposed classification scheme allows for the categorization of the fault diagnosis modules. The concept is implemented on AD shuttle buses and evaluated to demonstrate its capabilities.
BibTeX
@inproceedings{orf2024modular,
  author = {Stefan Orf and Sven Ochs and Jens Doll and Albert Schotschneider and Marc Heinrich and Marc René Zofka and J. Marius Zöllner},
  title = {{Modular Fault Diagnosis Framework for Complex Autonomous Driving Systems}},
  booktitle = {ICCP},
  year = {2024},
  url = {https://openreview.net/forum?id=r0onNgcDx9},
}

Loading related content...

Figure