On Model Reconciliation: How to Reconcile When Robot Does not Know Human's Model?

Ho Tuan Dung, Tran Cao Son

TL;DR

This paper introduces a dialogue-based framework for Model Reconciliation, allowing an AI agent (robot) to explain its plans to a human even when it doesn't know the human's internal mental model. By treating explanation as a back-and-forth conversation rather than a one-way data dump, the robot can iteratively discover model discrepancies and reach an agreement on the optimal plan.

What Problem Does This Paper Address?

In Explainable AI (XAI) planning, the Model Reconciliation Problem (MRP) occurs when a robot’s optimal plan seems wrong or sub-optimal to a human because their internal "models" of the world differ (e.g., the human thinks a certain action requires a tool the robot knows isn't needed).

Traditional solutions to MRP assume the robot is a mind-reader: it is expected to know exactly what the human believes so it can point out the specific differences. In the real world, this is rarely true. Humans update their knowledge, forget constraints, or have specialized expertise the robot hasn't seen. This paper tackles the challenge of reconciliation under model uncertainty, where the robot must explain its plan without prior knowledge of the human’s beliefs.

Methods

The authors propose a dialogue-based approach where the robot and human engage in a sequence of "rounds."

  1. Proposals: The robot suggests a plan and a potential explanation (a set of model changes).
  2. Responses: The human evaluates the proposal against their own model and responds with one of four types of feedback:
  3. Acceptance: The plan is optimal.
  4. Inapplicability: The explanation contains redundant or incorrect information.
  5. Inexecutability: The plan won't work in the human's model (and why).
  6. Sub-optimality: The human knows a "better" (shorter) plan.

The system is implemented using Answer Set Programming (ASP), specifically leveraging the "multi-shot" feature of the clingo solver. This allows the robot to maintain a stateful representation of the dialogue and efficiently update its search for the next proposal based on the human's feedback.

Datasets

The researchers evaluated their system using three classic AI planning benchmarks:

  • BLOCKSWORLD: The iconic "stacking blocks" domain.
  • LOGISTICS: Moving packages between locations using trucks and planes.
  • ROVER: Navigating a planetary surface to collect samples.

To simulate model discrepancies, the authors systematically modified the standard models to create "human" versions by randomly removing preconditions, deleting postconditions, or altering initial states.

Key Insights and Ideas

  • Explanation as Discovery: Instead of the robot "teaching" the human, the dialogue acts as a mutual discovery process. The robot learns just enough about the human's model to provide a valid explanation.
  • Minimalism via Priority: The robot uses a priority queue to suggest the shortest possible explanations first. This ensures that the final agreement is as concise as possible, respecting the human's cognitive load.
  • Truthfulness and Perfection: The framework assumes both parties are "truthful" and communication is "perfect." While idealized, these constraints provide a formal foundation for proving that the dialogue will always terminate with a correct explanation if one exists.
  • The "Why" Atom: The authors introduced a specific why(a, f, v, t) predicate to communicate why an action failed (e.g., "Action a failed at time t because fluent f had value v"). This provides the symbolic "glue" for the dialogue.

Key Results

  • Correctness: The dialogue-based system consistently found the same minimal explanations as state-of-the-art "model-aware" systems (like those by Chakraborti et al.).
  • Efficiency Trade-offs: As expected, the system is slower than model-aware versions. The bottleneck is the "Solve" call; the robot must re-run its planner multiple times as it receives new information from the human.
  • Non-Linear Complexity: Interestingly, the number of dialogue rounds (r) is not always proportional to the size of the explanation (|ε|). Some complex model differences can be resolved in a single round, while simple ones might require several back-and-forth exchanges to pinpoint.

Limitations and Future Work

  • Computational Overhead: Because the robot has to call a planner in every round, the approach can be slow for very large domains. The authors suggest that identifying better strategies for selecting proposals (rather than brute-force search) could speed up convergence.
  • Human Factors: The current model assumes a "perfect" human who can perfectly simulate their own planning model. Real-world humans might be inconsistent or unable to explain why they think a plan is sub-optimal.
  • Model-Free Extensions: While this paper assumes models are represented as logic programs, the authors note that future work could explore entirely model-free reconciliation using reinforcement learning, though this would likely require many more interactions.

Summary updated

BibTeX
@inproceedings{dung2022model,
  author = {Ho Tuan Dung and Tran Cao Son},
  title = {{On Model Reconciliation: How to Reconcile When Robot Does not Know Human's Model?}},
  booktitle = {ICLP Technical Communications / Doctoral Consortium},
  year = {2022},
  url = {https://openreview.net/forum?id=hFTbDoFxZB},
}

Loading related content...

Figure