On Model Reconciliation: How to Reconcile When Robot Does not Know Human's Model?

Ho Tuan Dung, Tran Cao Son

Abstract

This paper presents a dialogue-based framework for Model Reconciliation (MRP) in AI planning. Unlike previous approaches that assume the robot knows the human's mental model, this method allows a robot to iteratively discover model discrepancies through back-and-forth communication. By leveraging Answer Set Programming (ASP), the robot can provide minimal, truthful explanations that reconcile its optimal plan with the human's beliefs, even under significant model uncertainty.

The Challenge of Model Reconciliation

In Explainable AI (XAI), the Model Reconciliation Problem (MRP) arises when a robot's optimal plan appears incorrect or sub-optimal to a human observer. This discrepancy typically stems from differences in their internal "models" of the world—for instance, a human might believe a certain action requires a tool that the robot knows is unnecessary.

Traditional solutions to MRP often rely on the assumption that the robot has full knowledge of the human's mental model. In practice, however, humans frequently update their knowledge, forget constraints, or possess specialized expertise unknown to the robot. This research addresses the challenge of reconciliation under model uncertainty, where the robot must explain its plan without prior knowledge of the human’s specific beliefs.

A Dialogue-Based Approach

The authors propose a stateful, dialogue-based framework where the robot and human engage in sequential rounds of communication to resolve model differences. The process follows a structured interaction:

  1. Proposals: The robot suggests a plan along with a potential explanation (a set of model changes).
  2. Responses: The human evaluates the proposal against their own model and provides one of four types of feedback:
  3. Acceptance: The plan is confirmed as optimal.
  4. Inapplicability: The explanation contains redundant or incorrect information.
  5. Inexecutability: The plan is deemed unworkable in the human's model, accompanied by a reason.
  6. Sub-optimality: The human identifies a shorter or more efficient plan.

This system is implemented using Answer Set Programming (ASP), specifically utilizing the "multi-shot" feature of the clingo solver. This allows the robot to maintain the dialogue's state and efficiently refine its search for the next proposal based on human feedback. A key technical contribution is the introduction of the why(a, f, v, t) predicate, which serves as symbolic "glue" to communicate exactly why an action failed (e.g., "Action a failed at time t because fluent f had value v").

Experimental Evaluation

The researchers evaluated the framework using three classic AI planning benchmarks: BLOCKSWORLD (stacking blocks), LOGISTICS (moving packages), and ROVER (planetary exploration). To simulate real-world discrepancies, the authors created "human" versions of these models by randomly altering preconditions, postconditions, or initial states.

Figure 1: A basic BlocksWorld problem illustrating an initial state, a goal state, and the sequence of actions (plan) required to achieve the goal.

Key Insights and Results

  • Explanation as Discovery: The dialogue acts as a mutual discovery process. Rather than simply "teaching" the human, the robot learns just enough about the human's model to provide a valid, minimal explanation.
  • Minimalism and Cognitive Load: By using a priority queue to suggest the shortest possible explanations first, the robot ensures that the final agreement respects the human's cognitive load.
  • Correctness and Termination: Under the assumptions of truthfulness and perfect communication, the framework is proven to always terminate with a correct explanation if one exists.
  • Performance: While the system consistently finds the same minimal explanations as "model-aware" systems, it faces a computational trade-off. The need to re-run the planner multiple times during the dialogue makes it slower than non-interactive methods, with the number of dialogue rounds not always being strictly proportional to the complexity of the model difference.

Limitations and Future Work

While robust, the approach faces computational overhead in very large domains. Future work may explore more sophisticated proposal selection strategies to speed up convergence. Additionally, the current model assumes a "perfect" human agent; real-world applications will need to account for human inconsistency or the inability to articulate why a plan feels sub-optimal. Finally, extending this logic-based approach to model-free environments using reinforcement learning remains an open area for research.

Summary updated

BibTeX
@inproceedings{dung2022model,
  author = {Ho Tuan Dung and Tran Cao Son},
  title = {{On Model Reconciliation: How to Reconcile When Robot Does not Know Human's Model?}},
  booktitle = {ICLP Technical Communications / Doctoral Consortium},
  year = {2022},
  url = {https://openreview.net/forum?id=hFTbDoFxZB},
}
Figure