Model Reconciliation for Human-Aware Planning

This direction is merged into another direction: model reconciliation. Its last version stays readable and citable here.

Version 1 of 123 papers24 conceptsfrozen 2026-10-04

When an AI agent makes a decision that surprises a human user, the confusion usually stems from a mismatch in their internal models. The human might believe a certain action is impossible, or they might be unaware of a specific constraint the robot must follow. Model Reconciliation is the process of identifying these discrepancies and providing explanations that align the human’s mental model with the agent’s actual model.

The Problem of Mental Model Mismatch

In traditional automated planning, an agent finds a sequence of actions to reach a goal based on its internal model ($M_R$). However, in human-aware AI, we must also consider the human's model of the agent ($M_H$). As Sarath Sreedharan, Tathagata Chakraborti, and Subbarao Kambhampati argue in The Emerging Landscape of Explainable AI Planning and Decision Making, a plan that is optimal in $M_R$ might appear suboptimal or even invalid in $M_H$.

An explanation, in this context, is not just a trace of the agent's logic. Instead, it is a set of updates to the human's model—clarifying capabilities, costs, or constraints—so that the agent's plan becomes "optimal" in the eyes of the human.

Contrastive Explanations and Foils

Humans rarely ask for a full technical breakdown; instead, they ask, "Why did you do A instead of B?" The alternative B is known as a foil.

Benjamin Krarup and Daniele Magazzeni demonstrate in Contrastive Explanations of Plans through Model Restrictions that by treating the foil as a set of constraints, the agent can pinpoint exactly why the human's suggested alternative is inferior. This allows the system to generate a contrastive explanation that specifically addresses the user's misconception. Systems like RADAR-X turn this into an interactive loop, using these foils to elicit user preferences and suggest better plans.

Bridging the Knowledge Gap

A major hurdle in this field is that the agent often does not know what the human believes. Most early frameworks assumed the human's model $M_H$ was known, but recent research focuses on inferring this latent state:

From Explanation to Alignment

This research direction ultimately merges into the broader "Value Alignment" problem. In Expectation Alignment (EAL), Malek Mechergui and Sarath Sreedharan treat rewards as noisy communications of behavioral expectations. By aligning these expectations through interaction, the agent ensures it isn't just following a literal reward function, but is actually doing what the human intended.

Papers in this version

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_model_reconciliation_for_human_aware_pla_9ba0a14d,
  title = {Model Reconciliation for Human-Aware Planning},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/model-reconciliation-for-human-aware-planning/rdr_9ba0a14db59544fc9de5906cd8178022},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 1, frozen text):

@misc{lacuna_model_reconciliation_for_human_aware_pla_9ba0a14d_v1,
  title = {Model Reconciliation for Human-Aware Planning},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/model-reconciliation-for-human-aware-planning/rdr_9ba0a14db59544fc9de5906cd8178022/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}