Progressive Explanation Generation for Human-robot Teaming

Yu Zhang, Mehrdad Zakershahrak

Abstract: Generating explanations to justify behavior is an essential capability for a robotic teammate. Explanations help human partners better understand the situation and maintain trust. Prior work on robot explanation generation focuses on providing the reasoning behind decision-making from the explainer's perspective, often ignoring the cognitive requirements of the explainee. In this work, we address the "progressiveness" of explanations. A progressive explanation improves understanding by limiting the cognitive effort required at each step. We present a general formulation of Progressive Explanation Generation (PEG) and provide algorithms based on alternative quantifications of cognitive effort, evaluated in a standard planning competition domain.

The Challenge of Model Reconciliation

In human-robot teaming, trust depends on the human understanding why a robot is performing a specific action. Most current Explainable AI (XAI) research focuses on "Model Reconciliation"—identifying the specific differences between the robot’s internal model ($M^R$) and the human’s mental model ($M^H$) and providing the necessary information to resolve the discrepancy.

The Model Reconciliation setting: the robot ($M^R$) must bridge the gap between its policy and the human's mental model ($M^H$).

However, providing the correct information is only half the battle. If a robot delivers a massive, complex explanation all at once, or presents facts in a jarring sequence, the human's cognitive effort spikes. This paper addresses this "dissonance" by treating explanation as a sequential process where the goal is to make the transition from the human's initial understanding to the robot's correct understanding as seamless as possible.

Defining Progressiveness in Explanations

The authors formalize PEG as a search problem in "model space." Instead of just finding the shortest set of facts to explain a plan, they look for an ordered sequence of unit changes that minimizes a "model-plan distance metric." This metric serves as a proxy for cognitive effort.

Key technical components include:

  • Distance Metrics: They propose several ways to quantify effort, such as Plan Cost Difference (how much the perceived cost of the optimal plan changes) and Plan Editing Distance (how many steps in the actual plan change).
  • A* Search with Heuristics: Specialized admissible heuristics allow the robot to quickly find the "smoothest" explanation path.
A comparison of progressive vs. non-progressive explanations. The progressive path (triangles) avoids the sharp, jarring changes in plan cost seen in the non-progressive path (circles).

As shown in the figure above, a non-progressive explanation might keep the human in the dark for several steps before delivering a "shocker" that radically changes their understanding of the plan cost. A progressive explanation "nudges" the mental model gradually, leading to a smoother cognitive experience.

Experimental Evaluation

The approach was evaluated using the Rover domain, a standard benchmark from the International Planning Competition (IPC). The authors simulated human-robot teams where the "human" model was missing various features (preconditions, effects, or costs) compared to the robot's model.

Key Results

  • Lower Cumulative Cognitive Load: PEG explanations consistently resulted in much lower cumulative distance values compared to standard concise explanations. This suggests that the sequence of information significantly impacts how easily a human can process updates.
  • Efficiency: Despite the added complexity of sequencing, the A* search with the proposed heuristics found solutions in reasonable timeframes (often under a minute for complex rover problems), making it potentially viable for real-time interaction.
  • Minimal Overhead: PEG explanations were often nearly as short as the most concise explanations, proving that you can be "smooth" without being unnecessarily wordy.

Conclusion and Future Work

This work highlights that for explanations to be effective, they must be easy to follow, not just factually correct. By prioritizing "smoothness" over simple conciseness, robots can better support their human teammates' cognitive processes.

The authors acknowledge that the current "cognitive effort" metrics are mathematical proxies. Future work will involve user studies with actual humans to validate these metrics and expanding the framework to handle interactive dialogues where the robot can adjust its explanation based on real-time human feedback.

Summary updated

Abstract

Generating explanation to explain its behavior is an essential capability for a robotic teammate. Explanations help human partners better understand the situation and maintain trust of their teammates. Prior work on robot generating explanations focuses on providing the reasoning behind its decision making. These approaches, however, fail to heed the cognitive requirement of understanding an explanation. In other words, while they provide the right explanations from the explainer's perspective, the explainee part of the equation is ignored. In this work, we address an important aspect along this direction that contributes to a better understanding of a given explanation, which we refer to as the progressiveness of explanations. A progressive explanation improves understanding by limiting the cognitive effort required at each step of making the explanation. As a result, such explanations are expected to be smoother and hence easier to understand. A general formulation of progressive explanation is presented. Algorithms are provided based on several alternative quantifications of cognitive effort as an explanation is being made, which are evaluated in a standard planning competition domain.
BibTeX
@misc{zhang2019progressive,
  author = {Yu Zhang and Mehrdad Zakershahrak},
  title = {{Progressive Explanation Generation for Human-robot Teaming}},
  year = {2019},
  eprint = {1902.00604},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1902.00604},
}
Figure