Online Explanation Generation for Planning Tasks in Human-Robot Teaming

Mehrdad Zakershahrak, Ze Gong, Nikhillesh Sadassivam, Yu ZhangIROS 2020 · OpenReview

TL;DR

This paper introduces "online explanation generation" (OEG) for AI agents, breaking down complex explanations into smaller, digestible parts delivered incrementally during task execution. This approach significantly reduces human cognitive load and improves trust in human-robot teams, especially in demanding tasks, compared to traditional methods that present all information at once.

What Problem Does This Paper Address?

As AI agents, particularly robots, become more integrated into our lives, especially in collaborative settings like human-robot teaming, their ability to explain their decisions and behaviors is crucial for building trust and maintaining shared situation awareness. Existing "explainable AI" (XAI) methods often focus on generating accurate explanations but overlook a critical aspect: the human's cognitive capacity to understand these explanations. When explanations are complex or delivered all at once, they can overwhelm the human teammate, leading to increased mental workload, reduced performance, and a breakdown in effective collaboration. This paper tackles the challenge of making explanations more cognitively accessible by delivering them in an "online" fashion, spread out over time, to reduce the burden on the human.

Methods

The core innovation is the formulation of "online explanation generation" (OEG). Instead of providing a single, comprehensive explanation, OEG breaks it down into multiple "sub-explanations" that are delivered incrementally and intertwined with the robot's ongoing plan execution. The authors propose three variations of OEG:

  • OEG for Matching Plan Prefix (OEG-PP): This method ensures that each sub-explanation, when combined with previous ones, explains an increasingly longer prefix of the robot's plan. The goal is to make the human's understanding of the robot's plan consistent with the robot's actual plan prefix at each step. This is achieved through a model space search that ensures earlier sub-explanations don't create inconsistencies later.
  • OEG for Matching Next Action (OEG-NA): This is a more relaxed approach where each sub-explanation only needs to make the very next action interpretable. It assumes that humans in cognitively demanding tasks focus more on the immediate situation and less on retrospective evaluation.
  • OEG for Matching Any Prefix (OEG-AP): This variation addresses the assumption that optimal plans are always unique. It relaxes this constraint, allowing for explanations that match any of the potentially multiple optimal plans under the human's model.

These methods are built upon a "model reconciliation" framework, where the robot's internal model of the world is reconciled with the human's interpretation of that model.

Datasets

The paper evaluates its methods using two standard IPC (International Planning Competition) domains: Rover and Barman. For the human study, a modified rover domain was created, simulating a scenario where a rover collects and analyzes samples on Mars. This modified domain introduced specific information gaps (limited storage, camera calibration needs) to create model differences between the robot and the human.

Key Insights and Ideas

  • Cognitive Load is Key: The most significant insight is that the delivery of an explanation is as important as its content. Spreading information out incrementally dramatically reduces the mental effort required for humans to process it.
  • Online is Better for Demanding Tasks: For tasks that inherently require high cognitive effort, presenting explanations piecemeal is not just helpful, but essential for effective human-robot collaboration.
  • Breaking Down Complexity Works: The OEG methods demonstrate that complex explanations can indeed be decomposed into smaller, manageable parts without losing their explanatory power.
  • Model Reconciliation is a Powerful Framework: The underlying model reconciliation approach provides a robust foundation for generating explanations that address differences in understanding between humans and AI.
  • Relaxing Uniqueness Helps: The OEG-AP method, by not assuming a unique optimal plan, shows potential for generating more concise explanations, suggesting that flexibility in interpretation can be beneficial.

Key Results

  • Reduced Mental Workload: Human studies showed that OEG methods significantly reduced perceived mental workload (measured by NASA TLX) compared to traditional explanations (MCE).
  • Improved Task Performance: Subjects using OEG explanations were more accurate in identifying questionable robot actions and exhibited higher overall accuracy in their task performance.
  • Increased Trust: The number of questionable actions identified by subjects was significantly lower for OEG methods, indicating increased trust in the robot's behavior.
  • Computational Efficiency: Synthetically, OEG methods were generally faster than baseline MCE methods, attributed to the reduced search space in incremental generation.
  • Explanation Size: While the total size of sub-explanations in OEG could be larger than a single MCE, the average size of each sub-explanation was significantly smaller, making them more digestible.

Limitations and Future Work

  • Computational Cost of OEG-PP: The paper acknowledges that the model space search in OEG-PP can be computationally expensive, leading them to implement an approximate forward search method. However, this approximation means that for OEG-PP, explanations are created offline rather than truly online.
  • OEG-NA's Limited Scope: The OEG-NA method, while simple, might be too simplistic for scenarios requiring a deeper understanding of the robot's overall plan.
  • Future Work: The authors suggest exploring methods that lie between OEG-PP and OEG-NA to find an optimal balance. They also plan to investigate how to best combine the benefits of different OEG variations and further explore the impact of non-unique optimal plans.

Summary updated

BibTeX
@inproceedings{zakershahrak2020online,
  author = {Mehrdad Zakershahrak and Ze Gong and Nikhillesh Sadassivam and Yu Zhang},
  title = {{Online Explanation Generation for Planning Tasks in Human-Robot Teaming}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2020},
  url = {https://openreview.net/forum?id=rBoCpU85dp_},
}
Figure