When an intelligent agent forms an initial hypothesis, it constructs an internal mental model of its environment. Ideally, when new, contradictory evidence arrives, the agent should dynamically update this model. However, a pervasive failure mode across both classical symbolic AI and modern large language models (LLMs) is belief inertia or anchoring bias: the tendency to stubbornly cling to initial assumptions, even when they are directly disproven by subsequent data.
Understanding why intelligent systems fail to revise their beliefs requires looking at how contradictions disrupt both probabilistic frameworks and deep learning architectures.
The Mechanism of Stubbornness
Why do models fail to revise their beliefs? In probabilistic and Bayesian frameworks, the issue often stems from how evidence accumulates over time. As an agent gathers data, its certainty in a specific hypothesis grows, creating a heavy mathematical "prior."
Arnav Jhala et al. describe this exact phenomenon in Tuning Belief Revision for Coordination with Inconsistent Teammates. When an AI agent tracks a human teammate whose goals suddenly shift, standard Bayesian updates adapt too slowly. The accumulated historical evidence creates a high level of certainty that becomes incredibly difficult to overturn, causing the agent to misinterpret the teammate's new actions.
In large language models, belief inertia manifests not as mathematical weight, but as belief entrenchment and rationalization. Instead of acting as neutral observers when presented with new data, LLMs frequently use their generation capabilities to "double down" on and justify their initial guesses.
To detect this deviation from optimal reasoning, Zhonghao He et al. introduced the Martingale Score in Martingale Score: An Unsupervised Metric for Bayesian Rationality in LLM Reasoning. By measuring how predictable a model's belief updates are, the authors demonstrated that a model's tendency to rationalize its prior biases strongly correlates with a drop in overall reasoning accuracy.
[Initial Evidence] ---> [Initial Hypothesis Formed]
|
v
[Contradictory Data] -> [Belief Inertia / Anchoring] ---> [Rationalization / Double Down]
Anchoring in Multi-Turn and Interactive Environments
Belief inertia becomes particularly disruptive in interactive settings where an agent must track evolving states over a multi-turn conversation.
Clinical Diagnosis and Premature Closure
In complex domains like medicine, anchoring can lead to catastrophic failures. Christopher Chiu et al. evaluated this in Simulating Viva Voce Examinations to Evaluate Clinical Reasoning in Large Language Models, finding that LLMs mimic human cognitive errors like "premature closure." Once a model fixates on an initial diagnostic hypothesis, it struggles to perform dynamic belief revision when subsequent turns introduce conflicting clinical data.
This limitation is echoed by Miao Jing et al. in Beyond Classification Accuracy: Neural-MedBench and the Need for Deeper Reasoning Benchmarks. Their findings show that while models excel at static medical image classification, their performance plummets in multi-turn consultations because they fail to discard incorrect initial hypotheses when faced with disconfirming clinical evidence.
Tool Use and Sequential Assistants
In everyday assistant tasks, anchoring causes models to lose track of user intent. When a user changes their mind mid-interaction, the model must update its internal state tracking. However, as Jun Wang et al. discovered in HammerBench: Fine-Grained Function-Calling Evaluation in Real Mobile Assistant Scenarios, many language models exhibit a persistence bias. When a user explicitly corrects or changes an instruction, the model frequently retains outdated parameters from the earlier part of the conversation, failing to execute the new command accurately.
Trapped by Priors in Scientific Discovery
When agents are deployed to discover rules, code, or physical laws through trial and error, belief persistence acts as a barrier to iteration.
- Logical Puzzles: Kaiyu He et al. studied this in IDEA: Enhancing the Rule Learning Ability of Large Language Model Agent through Induction, Deduction, and Abduction. They noted that a primary bottleneck in interactive problem-solving is an agent's reluctance to overhaul an initial hypothesis when environmental feedback contradicts it, leading the agent to perform redundant, unhelpful actions.
- Physical Environments: This stubbornness persists even when agents run physical experiments. In PhysGym: Benchmarking LLMs in Interactive Physics Discovery with Controlled Priors, Yimeng Chen et al. demonstrated that autonomous agents frequently fail to refine their formulas based on empirical data, remaining "trapped" by their initial incorrect assumptions.
- Code Optimization: Similarly, in software engineering tasks, Yuze Zhao et al. identified a form of "cognitive stubbornness" in Unveiling the Magic of Code Reasoning through Hypothesis Decomposition and Amendment, where models refuse to abandon an incorrect initial coding framework despite receiving explicit negative feedback or error logs.
Memory Pollution and World Representations
Belief inertia also corrupts how agents maintain long-term memory and track external states, such as spatial layouts or the mental states of others.
Memory Pollution
When an agent misinterprets data early in a long context, that mistake can permanently poison its memory state. Yaorui Shi et al. document this in Look Back to Reason Forward: Revisitable Memory for Long-Context LLM Agents. Once an agent commits an initial hallucination or misinterpretation to its memory, it struggles to self-correct, prioritizing the polluted memory over new, conflicting evidence.
Spatial Tracking
This failure to overwrite obsolete beliefs is highly visible in dynamic environments. In Theory of Space: Can Foundation Models Construct Spatial Beliefs through Active Exploration?, Pingyue Zhang et al. found that foundation models stubbornly cling to outdated spatial maps. Even after actively exploring and seeing that a physical environment has changed, the models continue to report the old layout, demonstrating a lack of robust mechanisms for overwriting obsolete beliefs.
Theory of Mind
A parallel issue occurs in social reasoning. When tracking the beliefs of characters in a narrative, LLMs struggle to separate prior explicit statements from subsequent logical implications. Kanishk Gandhi et al. demonstrated this in Understanding Social Reasoning in Language Models with Language Models. If a character's initial belief is explicitly stated in the text, the model anchors to it, failing to update that character's internal state when subsequent causal events logically alter what that character should know.
The Formal Challenge of Contradiction
The difficulty of handling direct contradictions is not unique to modern deep learning; it has long been a fundamental challenge in formal logic and epistemic planning. Traditional logical systems are built on consistency; when an observation directly violates an internal rule or hard constraint, standard models often break down or become logically inconsistent.
[Standard Epistemic Logic] + [Contradictory Observation] ---> Logical Inconsistency / System Failure
[Retrospective Revision] + [Contradictory Observation] ---> Trigger History Rewind & Re-evaluate
As Kaile Su et al. observed in Observation-based Model for BDI-Agents, abstract Belief-Desire-Intention (BDI) logics frequently fail to provide a mechanism for maintaining consistency when an agent's real-world sensor observations directly contradict its internal mental state. This limitation makes it difficult to model agents over time.
Ronen Brafman et al. highlighted a similar issue in Towards Action Prediction Using a Mental-Level Model, noting that predictive frameworks struggle to anticipate an agent's subsequent actions once the system enters a "revision state" caused by an unexpected, belief-violating observation.
To resolve this, formal methods must treat contradictions not as system failures, but as triggers for retrospective re-evaluation. Aaron Hunter 0001 et al. addressed this by developing the action description language $\mathcal{A}_B$ in An Action Description Language for Iterated Belief Change, which explicitly formalizes iterated belief changes, allowing an agent to rewrite its perceived history when a sensing result exposes a false assumption.
Despite these theoretical advancements, practical implementations often omit these computationally expensive revision mechanisms:
- Epistemic Planning: Thorsten Engesser et al. point out in Towards Epistemic-Doxastic Planning with Observation and Revision that many lightweight epistemic logics remain entirely unable to model belief revision, leaving agents incapable of recovering from misinformation or tracking unobserved environmental changes.
- Probability Kinematics: In probabilistic reasoning, Michele Caprio et al. note in Dynamic precise and imprecise probability kinematics that standard models for updating subjective probabilities assume new evidence is inherently consistent with the existing knowledge base. They lack robust mathematical mechanisms for handling incoming data that directly violates previously established certainties or hard constraints.
Whether designing symbolic planners or prompting large language models, bridging this "active-passive gap" requires moving away from static state tracking. To build truly adaptive agents, AI systems must be engineered to treat contradictory evidence not as an anomaly to be rationalized away, but as a fundamental signal to tear down and rebuild their internal models of the world.
Key Foundational Work
Towards Action Prediction Using a Mental-Level Model (1995) — contributed 2 concept elements to this direction.
Observation-based Model for BDI-Agents (2005) — contributed 1 concept elements to this direction.