Abstract
Vision-based autonomous controllers often struggle with out-of-distribution (OOD) inputs, leading to potential safety violations. This paper proposes a novel framework that shifts focus from component-level errors to system-level anomalies. By integrating Hamilton-Jacobi reachability analysis with deep learning, the framework identifies states that inevitably lead to failure, automatically generates labeled training data, and deploys a runtime monitor to trigger safety-preserving fallback mechanisms.
Introduction
Autonomous systems, such as self-driving cars and drones, increasingly rely on vision-based controllers. A significant challenge for these systems is handling OOD inputs. Traditional monitoring often focuses on individual components (e.g., image blurriness). However, the authors argue that system-level monitoring is more critical: an input is truly "anomalous" only if it leads to a violation of the robot's safety constraints within its specific closed-loop dynamics. This paper introduces a method to identify these system-level anomalies by analyzing the consequences of errors rather than just their magnitude.
System-Level Anomaly Detection Framework
The proposed framework operates in two distinct phases: offline training and online monitoring.
- Offline Failure Mining: Hamilton-Jacobi (HJ) Reachability analysis is used to compute a Backward Reachable Tube (BRT). The BRT defines the set of states from which a failure is guaranteed under the current vision-based controller.
- Automatic Data Labeling: A photorealistic simulator (X-Plane) renders images for states inside and outside the BRT, labeling them as "anomalous" or "safe," respectively. This eliminates the need for manual labeling of failure cases.
- Runtime Monitoring: A binary classifier (EfficientNet-B0) is trained on this dataset. During online operation, it flags anomalous inputs, prompting a switch to a fallback controller that prioritizes safety (e.g., by reducing aircraft velocity).
Experimental Evaluation
The framework was evaluated using the TaxiNet environment, which simulates an autonomous aircraft taxiing on a runway.
The training set included 240,000 images across 12 conditions, including different times of day, weather, and runway markings. Testing involved unseen airports (KSFO, KEWR) and lighting conditions to evaluate the detector's generalization capabilities. The state space was defined by the aircraft's position relative to the centerline ($p_x, p_y$) and its heading angle ($\theta$).
Key Insights and Results
The study revealed that safety is highly context-dependent. For instance, certain positions were deemed anomalous in daylight but safe at night due to the high contrast provided by runway lights, which aided the controller's accuracy. This highlights that the BRT-based approach captures the actual consequences of visual errors.
Key results include:
- High Reliability: The detector achieved accuracy between 89.7% and 98.1% and recall between 90.3% and 97.4% across various unseen environments.
- Safety Improvement: The integrated pipeline reduced the volume of failure states from 25.75% down to 18.24%, significantly shrinking the system's "danger zone."
- Real-time Performance: The monitor is lightweight enough to run at 25Hz, making it suitable for real-time deployment on robotic hardware.
Conclusion and Future Work
While effective, the current approach faces the "curse of dimensionality" in HJ Reachability, which typically struggles with high-dimensional state spaces. Future work aims to utilize sampling-based reachability methods (like DeepReach) and automate the semantic analysis of failure modes. Additionally, developing more sophisticated, "intelligent" fallback controllers remains a priority for enhancing autonomous resilience in complex environments.
Summary updated