TL;DR
This paper tackles the challenge of generating explanations for robot actions that are not just correct from the robot's perspective, but also understandable and cognitively efficient for humans. They propose a novel "progressive explanation generation" approach that learns the optimal order of presenting information by modeling human preferences through inverse reinforcement learning, leading to improved task performance and reduced cognitive load.
What Problem Does This Paper Address?
In human-robot teaming, robots need to explain their decisions to their human partners to build trust and ensure shared situation awareness. Existing methods often focus on providing the correct rationale from the robot's viewpoint, but fail to consider the explainee's cognitive capacity and how they best learn and understand information. This gap is particularly critical when explanations are complex and must be delivered sequentially. The authors highlight that simply providing the right information isn't enough; the order in which it's presented significantly impacts human comprehension and task performance. They aim to bridge this gap by developing a system that generates explanations in an order optimized for human understanding.
Methods
The core of the paper is the Progressive Explanation Generation (PEG) framework. This approach models the process of generating an explanation as a sequential update of the human's internal model of the robot's behavior.
- Goal-Based MDP Formulation: The problem of learning the optimal order of explanation components (unit feature changes) is framed as a goal-based Markov Decision Process (MDP). The states in this MDP represent the human's current model, and actions correspond to unit feature changes that update this model.
- Inverse Reinforcement Learning (IRL): To learn the human's preferences for information order, the authors employ IRL. They assume that the cognitive effort required to understand each unit feature change can be quantified as a reward signal. By collecting human-generated explanations (traces), they use MaxEnt-IRL to learn a model distance metric that captures this cognitive effort.
- Model Distance Metric: This metric quantifies the cognitive cost associated with understanding a unit feature change. It's learned as a linear combination of domain-dependent and domain-independent features. Domain-independent features, such as action distance and Levenshtein distance between plans, are particularly emphasized as they capture general aspects of plan changes.
Datasets
The authors conducted human-subject studies in two distinct domains:
- Scavenger-Hunt Domain: A simulated damaged office building scenario where a participant acts as a first responder guiding a trapped human. This domain involved 10 possible changes that could affect the trapped human's plan.
- Escape-Room Domain: A simulated damaged nuclear plant maze where a participant acts as an external agent informing an internal agent about dangerous locations. This domain involved 7 contingencies (unit feature changes).
Both domains were designed to be moderately complex, requiring non-trivial cognitive effort. The data collected from these studies (human-generated explanations) was used to train and evaluate the PEG approach.
Key Insights and Ideas
- Order Matters Fundamentally: The most striking insight is that the order of information in an explanation is not just a stylistic choice but a critical factor influencing human understanding and task performance. This is particularly true for complex tasks requiring sequential information processing.
- Dynamic Understanding: Human understanding of an explanation is not a static event that happens after the explanation is complete. Instead, humans actively "replan" and update their understanding during the explanation process, especially for moderately to highly complex tasks.
- Domain-Independent Preferences: While domain-specific features play a role, domain-independent features (like how much a plan changes) are surprisingly strong predictors of human preferences for explanation order. This suggests a general cognitive mechanism for processing sequential information.
- PEG Reduces Cognitive Load: By optimizing the order of information, PEG effectively reduces the cognitive effort required for humans to understand the robot's actions, leading to better performance and less mental strain.
Key Results
- High Accuracy in Learning Preferences: The PEG approach successfully learned human preferences for explanation order, achieving 85.2% accuracy in matching human-generated explanations in the scavenger-hunt domain.
- Improved Task Performance and Reduced Cognitive Load: In the escape-room domain, PEG significantly outperformed baseline methods (random order and Manhattan distance) across all NASA TLX metrics (mental demand, temporal demand, performance, effort, frustration). Objective metrics also showed PEG improved task accuracy.
- Validation of Hypotheses: The results validated the authors' hypotheses: (H1) their method can learn human preferences, (H2) humans replan dynamically during explanation, and (H3) progressive explanations reduce cognitive load and improve task performance.
Limitations and Future Work
- Markov Assumption: The current MDP formulation relies on a Markov assumption, which might be restrictive for modeling complex human cognition where long-term dependencies could be important.
- Unit Feature Changes: The current framework focuses on explaining one unit feature change at a time. Future work could explore generating "sub-explanations" that convey multiple related pieces of information simultaneously.
- Generalizing MDP: Relaxing the Markov assumption in the MDP model could lead to a more comprehensive representation of human cognition.
- Investigating Feature Impact: Further investigation into the impact of different feature sets on the learned model distance metric is suggested.
Summary updated