Order Matters: Generating Progressive Explanations for Planning Tasks in Human-Robot Teaming

Mehrdad Zakershahrak, Shashank Rao Marpally, Akshay Sharma, Ze Gong, Yu Zhang

TL;DR

This paper tackles the challenge of generating explanations for robot actions that are not just correct from the robot's perspective, but also understandable and cognitively efficient for humans. They propose a novel "progressive explanation generation" approach that learns the optimal order of presenting information by modeling human preferences through inverse reinforcement learning, leading to improved task performance and reduced cognitive load.

What Problem Does This Paper Address?

In human-robot teaming, robots need to explain their decisions to their human partners to build trust and ensure shared situation awareness. Existing methods often focus on providing the correct rationale from the robot's viewpoint, but fail to consider the explainee's cognitive capacity and how they best learn and understand information. This gap is particularly critical when explanations are complex and must be delivered sequentially. The authors highlight that simply providing the right information isn't enough; the order in which it's presented significantly impacts human comprehension and task performance. They aim to bridge this gap by developing a system that generates explanations in an order optimized for human understanding.

Methods

The core of the paper is the Progressive Explanation Generation (PEG) framework. This approach models the process of generating an explanation as a sequential update of the human's internal model of the robot's behavior.

  • Goal-Based MDP Formulation: The problem of learning the optimal order of explanation components (unit feature changes) is framed as a goal-based Markov Decision Process (MDP). The states in this MDP represent the human's current model, and actions correspond to unit feature changes that update this model.
  • Inverse Reinforcement Learning (IRL): To learn the human's preferences for information order, the authors employ IRL. They assume that the cognitive effort required to understand each unit feature change can be quantified as a reward signal. By collecting human-generated explanations (traces), they use MaxEnt-IRL to learn a model distance metric that captures this cognitive effort.
  • Model Distance Metric: This metric quantifies the cognitive cost associated with understanding a unit feature change. It's learned as a linear combination of domain-dependent and domain-independent features. Domain-independent features, such as action distance and Levenshtein distance between plans, are particularly emphasized as they capture general aspects of plan changes.

Datasets

The authors conducted human-subject studies in two distinct domains:

  • Scavenger-Hunt Domain: A simulated damaged office building scenario where a participant acts as a first responder guiding a trapped human. This domain involved 10 possible changes that could affect the trapped human's plan.
  • Escape-Room Domain: A simulated damaged nuclear plant maze where a participant acts as an external agent informing an internal agent about dangerous locations. This domain involved 7 contingencies (unit feature changes).

Both domains were designed to be moderately complex, requiring non-trivial cognitive effort. The data collected from these studies (human-generated explanations) was used to train and evaluate the PEG approach.

Key Insights and Ideas

  • Order Matters Fundamentally: The most striking insight is that the order of information in an explanation is not just a stylistic choice but a critical factor influencing human understanding and task performance. This is particularly true for complex tasks requiring sequential information processing.
  • Dynamic Understanding: Human understanding of an explanation is not a static event that happens after the explanation is complete. Instead, humans actively "replan" and update their understanding during the explanation process, especially for moderately to highly complex tasks.
  • Domain-Independent Preferences: While domain-specific features play a role, domain-independent features (like how much a plan changes) are surprisingly strong predictors of human preferences for explanation order. This suggests a general cognitive mechanism for processing sequential information.
  • PEG Reduces Cognitive Load: By optimizing the order of information, PEG effectively reduces the cognitive effort required for humans to understand the robot's actions, leading to better performance and less mental strain.

Key Results

  • High Accuracy in Learning Preferences: The PEG approach successfully learned human preferences for explanation order, achieving 85.2% accuracy in matching human-generated explanations in the scavenger-hunt domain.
  • Improved Task Performance and Reduced Cognitive Load: In the escape-room domain, PEG significantly outperformed baseline methods (random order and Manhattan distance) across all NASA TLX metrics (mental demand, temporal demand, performance, effort, frustration). Objective metrics also showed PEG improved task accuracy.
  • Validation of Hypotheses: The results validated the authors' hypotheses: (H1) their method can learn human preferences, (H2) humans replan dynamically during explanation, and (H3) progressive explanations reduce cognitive load and improve task performance.

Limitations and Future Work

  • Markov Assumption: The current MDP formulation relies on a Markov assumption, which might be restrictive for modeling complex human cognition where long-term dependencies could be important.
  • Unit Feature Changes: The current framework focuses on explaining one unit feature change at a time. Future work could explore generating "sub-explanations" that convey multiple related pieces of information simultaneously.
  • Generalizing MDP: Relaxing the Markov assumption in the MDP model could lead to a more comprehensive representation of human cognition.
  • Investigating Feature Impact: Further investigation into the impact of different feature sets on the learned model distance metric is suggested.

Summary updated

Abstract

Prior work on generating explanations in a planning and decision-making context has focused on providing the rationale behind an AI agent's decision making. While these methods provide the right explanations from the explainer's perspective, they fail to heed the cognitive requirement of understanding an explanation from the explainee's (the human's) perspective. In this work, we set out to address this issue by first considering the influence of information order in an explanation, or the progressiveness of explanations. Intuitively, progression builds later concepts on previous ones and is known to contribute to better learning. In this work, we aim to investigate similar effects during explanation generation when an explanation is broken into multiple parts that are communicated sequentially. The challenge here lies in modeling the humans' preferences for information order in receiving such explanations to assist understanding. Given this sequential process, a formulation based on goal-based MDP for generating progressive explanations is presented. The reward function of this MDP is learned via inverse reinforcement learning based on explanations that are retrieved via human subject studies. We first evaluated our approach on a scavenger-hunt domain to demonstrate its effectively in capturing the humans' preferences. Upon analyzing the results, it revealed something more fundamental: the preferences arise strongly from both domain dependent and independence features. The correlation with domain independent features pushed us to verify this result further in an escape room domain. Results confirmed our hypothesis that the process of understanding an explanation was a dynamic process. The human preference that reflected this aspect corresponded exactly to the progression for knowledge assimilation hidden deeper in our cognitive process.
BibTeX
@inproceedings{zakershahrak2021order,
  author = {Mehrdad Zakershahrak and Shashank Rao Marpally and Akshay Sharma and Ze Gong and Yu Zhang},
  title = {{Order Matters: Generating Progressive Explanations for Planning Tasks in Human-Robot Teaming}},
  booktitle = {IEEE International Conference on Robotics and Automation},
  year = {2021},
  url = {https://openreview.net/forum?id=KnfXRngc-U},
}
Figure