Balancing Explicability and Explanations in Human-Aware Planning

Tathagata Chakraborti, Sarath Sreedharan, Subbarao Kambhampati

Abstract

This paper introduces MEGA (Multi-model Explanation Generation Algorithm), the first planning framework to unify "acting as expected" (explicability) and "explaining why I'm acting differently" (explanation) into a single trade-off. In human-aware planning, a robot must often navigate the gap between its own model and a human's mental model. MEGA allows a robot to decide whether to follow a suboptimal but explicable path or an optimal path accompanied by a concise explanation. Through evaluations in IPC domains and a human study in a search-and-rescue scenario, the authors demonstrate that this balanced approach effectively manages communication costs and human expectations.

Introduction: The Mental Model Gap

In collaborative environments, a robot's actual model of the world ($M^R$) often differs from the human observer's mental model of the robot ($M^R_h$). This divergence can lead to confusion when the robot performs actions that seem suboptimal or nonsensical to the human. Traditionally, robots have addressed this in two ways:

  1. Explicable Planning: Choosing actions that align with the human's expectations, even at the cost of efficiency.
  2. Plan Explanation: Executing the robot's optimal plan and then providing a "model reconciliation" explanation to update the human's mental model.
A search-and-rescue scenario illustrating the mental model gap where optimal plans in the robot's model are not executable or optimal in the human's model.

As shown in the scenario above, a robot might know a path is clear while the human believes it is blocked. Following the "expected" path is inefficient, but taking the "new" path requires an explanation to reconcile the human's outdated model.

The MEGA Framework

The authors propose that explicability is a special case of explanation where the explanation length is zero. MEGA unifies these concepts by minimizing a weighted sum of the explanation length ($|E|$) and the robot's departure from its own optimal plan cost. A hyper-parameter, $\alpha$, serves as a "knob" to tune this balance:

  • Low $\alpha$: Prioritizes explicability (minimizing human confusion).
  • High $\alpha$: Prioritizes robot efficiency (minimizing plan cost).

MEGA searches through "model space" using an $A^*$ search to find the smallest set of model updates that makes a chosen plan appear optimal to the human. To handle the computational complexity of this search, the authors also introduced MEGA-approx, a faster version suitable for semi-online use.

Evaluation and Results

The framework was evaluated using modified International Planning Competition (IPC) domains (Rover and Barman) and a custom Urban Search and Rescue (USAR) scenario.

Algorithmic Trade-offs

In the Rover and Barman domains, MEGA successfully identified a Pareto frontier between plan cost and explanation length.

The Pareto frontier showing the trade-off between explicability cost and explanation length across different problem instances.

As illustrated in Figure 4 above, increasing the robot's willingness to explain (higher $\alpha$) allows it to reduce its own plan cost. Conversely, when explanations are costly or bandwidth is limited, the robot can opt for more explicable, albeit less efficient, plans.

Human Factors Study

A study using the USAR domain revealed that providing explicable plans significantly reduced the frequency of human requests for explanations. Interestingly, participants showed "risk-averse" behavior, requesting explanations even for explicable plans about 30% of the time in high-stakes scenarios. The study also noted bimodal preferences, suggesting that the $\alpha$ parameter may need to be personalized for different users' levels of trust and skepticism.

Conclusion and Future Work

MEGA provides a robust foundation for human-aware planning by treating explicability and explanation as a unified problem. While searching model space remains computationally challenging, the balanced approach offers a more flexible way for robots to interact with humans. Future work will explore learning the $\alpha$ parameter dynamically and incorporating "ontic" explanations—communicating model changes through actions rather than words.

Summary updated

Abstract

Human-aware planning involves generating plans that are explicable as well as providing explanations when such plans cannot be found. In this paper, we bring these two concepts together and show how an agent can achieve a trade-off between these two competing characteristics of a plan. In order to achieve this, we conceive a first of its kind planner MEGA that can augment the possibility of explaining a plan in the plan generation process itself. We situate our discussion in the context of recent work on explicable planning and explanation generation and illustrate these concepts in two well-known planning domains, as well as in a demonstration of a robot in a typical search and reconnaissance task. Human factor studies in the latter highlight the usefulness of the proposed approach.
BibTeX
@inproceedings{chakraborti2019balancing,
  author = {Tathagata Chakraborti and Sarath Sreedharan and Subbarao Kambhampati},
  title = {{Balancing Explicability and Explanations in Human-Aware Planning}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2019},
  doi = {10.24963/ijcai.2019/185},
}
Figure