Belief Inertia and Anchoring in AI Reasoning

38 concepts24 papers1 proposal

When an intelligent agent forms an initial hypothesis, it constructs an internal mental model of its environment. Ideally, when new, contradictory evidence arrives, the agent should dynamically update this model. However, a pervasive failure mode across both classical symbolic AI and modern large language models (LLMs) is belief inertia or anchoring bias: the tendency to stubbornly cling to initial assumptions, even when they are directly disproven by subsequent data.

Understanding why intelligent systems fail to revise their beliefs requires looking at how contradictions disrupt both probabilistic frameworks and deep learning architectures.

The Mechanism of Stubbornness

Why do models fail to revise their beliefs? In probabilistic and Bayesian frameworks, the issue often stems from how evidence accumulates over time. As an agent gathers data, its certainty in a specific hypothesis grows, creating a heavy mathematical "prior."

Arnav Jhala et al. describe this exact phenomenon in Tuning Belief Revision for Coordination with Inconsistent Teammates. When an AI agent tracks a human teammate whose goals suddenly shift, standard Bayesian updates adapt too slowly. The accumulated historical evidence creates a high level of certainty that becomes incredibly difficult to overturn, causing the agent to misinterpret the teammate's new actions.

In large language models, belief inertia manifests not as mathematical weight, but as belief entrenchment and rationalization. Instead of acting as neutral observers when presented with new data, LLMs frequently use their generation capabilities to "double down" on and justify their initial guesses.

To detect this deviation from optimal reasoning, Zhonghao He et al. introduced the Martingale Score in Martingale Score: An Unsupervised Metric for Bayesian Rationality in LLM Reasoning. By measuring how predictable a model's belief updates are, the authors demonstrated that a model's tendency to rationalize its prior biases strongly correlates with a drop in overall reasoning accuracy.

[Initial Evidence] ---> [Initial Hypothesis Formed]
                               |
                               v
[Contradictory Data] -> [Belief Inertia / Anchoring] ---> [Rationalization / Double Down]

Anchoring in Multi-Turn and Interactive Environments

Belief inertia becomes particularly disruptive in interactive settings where an agent must track evolving states over a multi-turn conversation.

Clinical Diagnosis and Premature Closure

In complex domains like medicine, anchoring can lead to catastrophic failures. Christopher Chiu et al. evaluated this in Simulating Viva Voce Examinations to Evaluate Clinical Reasoning in Large Language Models, finding that LLMs mimic human cognitive errors like "premature closure." Once a model fixates on an initial diagnostic hypothesis, it struggles to perform dynamic belief revision when subsequent turns introduce conflicting clinical data.

This limitation is echoed by Miao Jing et al. in Beyond Classification Accuracy: Neural-MedBench and the Need for Deeper Reasoning Benchmarks. Their findings show that while models excel at static medical image classification, their performance plummets in multi-turn consultations because they fail to discard incorrect initial hypotheses when faced with disconfirming clinical evidence.

Tool Use and Sequential Assistants

In everyday assistant tasks, anchoring causes models to lose track of user intent. When a user changes their mind mid-interaction, the model must update its internal state tracking. However, as Jun Wang et al. discovered in HammerBench: Fine-Grained Function-Calling Evaluation in Real Mobile Assistant Scenarios, many language models exhibit a persistence bias. When a user explicitly corrects or changes an instruction, the model frequently retains outdated parameters from the earlier part of the conversation, failing to execute the new command accurately.

Trapped by Priors in Scientific Discovery

When agents are deployed to discover rules, code, or physical laws through trial and error, belief persistence acts as a barrier to iteration.

Memory Pollution and World Representations

Belief inertia also corrupts how agents maintain long-term memory and track external states, such as spatial layouts or the mental states of others.

Memory Pollution

When an agent misinterprets data early in a long context, that mistake can permanently poison its memory state. Yaorui Shi et al. document this in Look Back to Reason Forward: Revisitable Memory for Long-Context LLM Agents. Once an agent commits an initial hallucination or misinterpretation to its memory, it struggles to self-correct, prioritizing the polluted memory over new, conflicting evidence.

Spatial Tracking

This failure to overwrite obsolete beliefs is highly visible in dynamic environments. In Theory of Space: Can Foundation Models Construct Spatial Beliefs through Active Exploration?, Pingyue Zhang et al. found that foundation models stubbornly cling to outdated spatial maps. Even after actively exploring and seeing that a physical environment has changed, the models continue to report the old layout, demonstrating a lack of robust mechanisms for overwriting obsolete beliefs.

Theory of Mind

A parallel issue occurs in social reasoning. When tracking the beliefs of characters in a narrative, LLMs struggle to separate prior explicit statements from subsequent logical implications. Kanishk Gandhi et al. demonstrated this in Understanding Social Reasoning in Language Models with Language Models. If a character's initial belief is explicitly stated in the text, the model anchors to it, failing to update that character's internal state when subsequent causal events logically alter what that character should know.

The Formal Challenge of Contradiction

The difficulty of handling direct contradictions is not unique to modern deep learning; it has long been a fundamental challenge in formal logic and epistemic planning. Traditional logical systems are built on consistency; when an observation directly violates an internal rule or hard constraint, standard models often break down or become logically inconsistent.

[Standard Epistemic Logic] + [Contradictory Observation] ---> Logical Inconsistency / System Failure
[Retrospective Revision]    + [Contradictory Observation] ---> Trigger History Rewind & Re-evaluate

As Kaile Su et al. observed in Observation-based Model for BDI-Agents, abstract Belief-Desire-Intention (BDI) logics frequently fail to provide a mechanism for maintaining consistency when an agent's real-world sensor observations directly contradict its internal mental state. This limitation makes it difficult to model agents over time.

Ronen Brafman et al. highlighted a similar issue in Towards Action Prediction Using a Mental-Level Model, noting that predictive frameworks struggle to anticipate an agent's subsequent actions once the system enters a "revision state" caused by an unexpected, belief-violating observation.

To resolve this, formal methods must treat contradictions not as system failures, but as triggers for retrospective re-evaluation. Aaron Hunter 0001 et al. addressed this by developing the action description language $\mathcal{A}_B$ in An Action Description Language for Iterated Belief Change, which explicitly formalizes iterated belief changes, allowing an agent to rewrite its perceived history when a sensing result exposes a false assumption.

Despite these theoretical advancements, practical implementations often omit these computationally expensive revision mechanisms:

  • Epistemic Planning: Thorsten Engesser et al. point out in Towards Epistemic-Doxastic Planning with Observation and Revision that many lightweight epistemic logics remain entirely unable to model belief revision, leaving agents incapable of recovering from misinformation or tracking unobserved environmental changes.
  • Probability Kinematics: In probabilistic reasoning, Michele Caprio et al. note in Dynamic precise and imprecise probability kinematics that standard models for updating subjective probabilities assume new evidence is inherently consistent with the existing knowledge base. They lack robust mathematical mechanisms for handling incoming data that directly violates previously established certainties or hard constraints.

Whether designing symbolic planners or prompting large language models, bridging this "active-passive gap" requires moving away from static state tracking. To build truly adaptive agents, AI systems must be engineered to treat contradictory evidence not as an anomaly to be rationalized away, but as a fundamental signal to tear down and rebuild their internal models of the world.

Key Foundational Work

Towards Action Prediction Using a Mental-Level Model (1995) — contributed 2 concept elements to this direction.
Observation-based Model for BDI-Agents (2005) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_belief_revision_failure_a7d5386a,
  title = {belief revision failure},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/belief-revision-failure/rdr_a7d5386ad9a44ccaba2a9d7aa681f5d2},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_belief_revision_failure_a7d5386a_v1,
  title = {belief revision failure},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/belief-revision-failure/rdr_a7d5386ad9a44ccaba2a9d7aa681f5d2/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake