Safe Handover in Mixed-Initiative Control for Cyber-Physical Systems

Frederik Wiehr, Anke Hirsch, Florian Daiber, Antonio Krüger, Alisa Kovtunova, Stefan Borgwardt, Ernie Chang, Vera Demberg, Marcel Steinmetz, Jörg HoffmannarXiv 2020 · arXiv

Introduction

Autonomous vehicles (AVs) promise to revolutionize transport, but the transition of control from machine to human—the "handover"—remains a critical safety bottleneck. When a driver is "out-of-the-loop," perhaps reading or resting, they lack the situational awareness needed to take over safely in an instant. Current systems often rely on abrupt auditory alerts that leave drivers stressed and confused. This paper proposes a novel framework that leverages formal AI methods to provide earlier warnings and clearer explanations, bridging the gap between automated systems and human cognition.

The Architecture of Safety

The proposed system integrates several advanced AI techniques into a unified architecture designed to anticipate hazards and communicate them effectively.

The proposed system architecture integrating formal reasoning with multimodal interaction.

The framework consists of four primary pillars:

  1. Automated Planning: Unlike reactive systems that only respond to immediate threats, this component simulates future states. It looks ahead to identify potential critical situations before they manifest, allowing the system to either navigate around them or initiate a timely handover.
  2. Description Logics (DL): By using formal ontologies, the system can represent complex road scenes in a way that mirrors human conceptual reasoning. This allows the car to "understand" why a situation is dangerous (e.g., identifying an "unsecured load" rather than just an "obstacle").
  3. Natural Language Generation (NLG): To avoid cognitive overload, the system translates abstract logical facts into concise, natural language. This ensures the driver receives actionable information rather than cryptic error codes.
  4. Multimodal HCI: The final realization of the handover uses both visual and verbal cues, ensuring the driver's attention is directed to the most relevant parts of the environment.

Timing and Explanation: The Two-Stage Warning

A key innovation of this research is the "two-stage" warning system. By extending the warning window, the system gives the driver more time to re-engage with the driving task.

Timeline of the proposed two-stage handover process compared to traditional alerts.

As illustrated in the timeline, the process begins at Point A, approximately 30 seconds before a predicted critical situation (such as a construction zone). This "heads-up" notification is triggered by the automated planner. At Point B, 15 seconds before the event, a final notification and auditory "beep" are issued. This 15-second buffer between the initial warning and the final handover request is crucial for allowing the driver to regain situational awareness gradually.

Furthermore, the system doesn't just tell the driver when to take over, but why. Instead of a generic alert, the driver might hear: "I am handing over control because there is a construction zone ahead with narrow lanes." This semantic clarity builds trust and helps the driver prioritize their visual search upon re-entering the loop.

Implementation and Evaluation

The framework was implemented using AirSim, a high-fidelity simulator for autonomous systems. The authors developed four distinct handover scenarios—ranging from static construction zones to dynamic "danger cars"—to test the system's efficacy.

While quantitative user study results are forthcoming, the proof-of-concept demonstrates that integrating formal reasoning with NLG is feasible within the tight time constraints of high-speed driving. Future evaluations will utilize the NASA-TLX to measure cognitive load and the Trust in Automation (TiA) questionnaire to assess how explanations impact driver confidence.

Conclusion and Future Work

This research highlights the importance of "explainable" handovers in cyber-physical systems. By combining the predictive power of automated planning with the semantic richness of description logics, the authors provide a roadmap for safer human-machine collaboration. Future work will focus on refining user models to tailor explanations to individual driver experience levels and addressing the challenges of real-world sensor noise that were not present in the simulated environment.

Summary updated

Abstract

For mixed-initiative control between cyber-physical systems (CPS) and its users, it is still an open question how machines can safely hand over control to humans. In this work, we propose a concept to provide technological support that uses formal methods from AI -- description logic (DL) and automated planning -- to predict more reliably when a hand-over is necessary, and to increase the advance notice for handovers by planning ahead of runtime. We combine this with methods from human-computer interaction (HCI) and natural language generation (NLG) to develop solutions for safe and smooth handovers and provide an example autonomous driving scenario. A study design is proposed with the assessment of qualitative feedback, cognitive load and trust in automation.
BibTeX
@misc{wiehr2020safe,
  author = {Frederik Wiehr and Anke Hirsch and Florian Daiber and Antonio Krüger and Alisa Kovtunova and Stefan Borgwardt and Ernie Chang and Vera Demberg and Marcel Steinmetz and Jörg Hoffmann},
  title = {{Safe Handover in Mixed-Initiative Control for Cyber-Physical Systems}},
  year = {2020},
  eprint = {2010.10967},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2010.10967},
}
Figure