Overcoming the Curse of Knowledge in Machine Theory of Mind

This direction is split into other directions: privileged information bias. Its last version stays readable and citable here.

Version 1 of 150 papers81 conceptsfrozen 2026-10-04

When humans interact, we constantly simulate the internal worlds of others. We understand that if Alice leaves the room before Bob moves her keys, Alice will still look for the keys where she last saw them. This ability to attribute mental states—beliefs, desires, and intentions—that differ from our own is known as Theory of Mind (ToM).

For Large Language Models (LLMs), this is a profound challenge. Despite their linguistic fluency, models frequently suffer from a "curse of knowledge" or an omniscience bias. Because the model has access to the entire text or "ground truth" of a scenario, it struggles to ignore that privileged information when predicting the behavior of a character who lacks it.

The Omniscience Trap

The fundamental problem, as identified by Neil C. Rabinowitz and colleagues in their work on Machine Theory of Mind, is that AI systems often assume other agents perceive and process the world exactly as the observer does. In a multi-party conversation, a model might see a full transcript. If a character leaves the room during a crucial update, the model often fails to "forget" that update when reasoning from that character's perspective.

This leads to a reality bias. As shown in Revisiting the Evaluation of Theory of Mind through Question Answering by Matthew Le and Maximilian Nickel, models default to reporting the current objective reality rather than the agent's potentially incorrect internal belief. The model "knows" the keys are in the drawer, so it predicts Alice will look in the drawer, even though Alice never saw them being moved.

Perspective-Taking as Information Filtering

To solve this, researchers have moved toward explicit perspective-taking mechanisms. The intuition is simple: if the model's "god's eye view" is the problem, we must force it to look through a narrower lens.

Alex Wilf and Paul Pu Liang introduced a framework called SIMTOM in Think Twice: Perspective-Taking Improves Large Language Models' Theory-of-Mind Capabilities. SIMTOM works in two stages:

  1. Context Filtering: The model is prompted to extract only the information a specific character has actually observed.
  2. Question Answering: The model answers the question using only that filtered context.

By physically removing the "spoiler" information from the prompt, the model is no longer tempted by the ground truth. A similar approach, PercepToM, developed by C. G. Jung and colleagues in Perceptions to Beliefs, focuses on identifying what a character perceived before inferring what they believe. This mimics human inhibitory control—the cognitive ability to suppress what we know to be true in order to reason about someone else's error.

Symbolic and Temporal Belief Tracking

While prompting helps, complex narratives with many characters and shifting beliefs require more robust "bookkeeping." Purely neural models often struggle to maintain distinct mental models for secondary characters, as noted by Maarten Sap and Yejin Choi in Neural Theory-of-Mind?.

To address this, researchers like Melanie Sclar and Sachin Kumar proposed SYMBOLICTOM in Minding Language Models' (Lack of) Theory of Mind. This system maintains explicit, recursive symbolic graphs of "who believes what about what." By treating beliefs as structured data that can be updated over time, the model can track information asymmetries without them leaking into one another.

Time is the critical dimension here. Guiyang Hou and colleagues argue in TimeToM that social reasoning is essentially a temporal problem. Their framework constructs "Temporal Belief State Chains," allowing the model to "travel back" to a point in time before a belief became false. This prevents the model from using future knowledge to explain past intentions.

The Application Gap

Even when models can correctly identify that a character holds a false belief, they often fail to use that knowledge to predict behavior. Yuling Gu and Oyvind Tafjord highlight this in SimpleToM, noting a massive "application gap." A model might correctly state "Alice thinks the keys are on the table," but when asked "Where will Alice look?", it might still answer "In the drawer" because it knows that's where they actually are.

This suggests that Theory of Mind in AI isn't just about better fact-tracking; it's about cognitive decoupling. The most successful recent directions, such as Decompose-ToM by Sneheel Sarangi, work by breaking down these "who thinks what" questions into isolated sub-tasks. By simulating each perspective in a vacuum, these systems finally begin to bridge the gap between knowing the truth and understanding a lie.

Papers in this version

History & citationVersion 1 · 2026-10-04

Split into other directions · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_overcoming_the_curse_of_knowledge_in_mac_171c1b83,
  title = {Overcoming the Curse of Knowledge in Machine Theory of Mind},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/overcoming-the-curse-of-knowledge-in-machine-theory-of-mind/rdr_171c1b83e23f4b1c8f0aed5afe290ef4},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_overcoming_the_curse_of_knowledge_in_mac_171c1b83_v1,
  title = {Overcoming the Curse of Knowledge in Machine Theory of Mind},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/overcoming-the-curse-of-knowledge-in-machine-theory-of-mind/rdr_171c1b83e23f4b1c8f0aed5afe290ef4/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}