Why Language Models Fail to Recognize Unsolvable States

18 concepts8 papers1 proposal

When humans tackle a problem, we constantly evaluate whether it is actually possible to solve. If we are asked to find a file on a computer without an internet connection or a file system tool, we stop immediately and report the missing capability. Large Language Models (LLMs), however, operate under a deep-seated solvability assumption: they behave as if every user request, mathematical state, or agent trajectory must have a valid path to success.

This fundamental blind spot manifests as a systematic optimism bias. Instead of recognizing an impossible task or a dead end, models will hallucinate solutions, force invalid logical transitions, or burn through computational resources on trajectories destined to fail.

Blindness to Dead Ends in Reasoning

Why do models struggle to see a dead end? In multi-step deduction, a model must evaluate not just the immediate next step, but whether that step leaves the problem in a state that is still solvable.

In latent space mathematical reasoning, Szegedy et al. found that while neural networks can successfully predict semantic transformations, they exhibit a distinct optimistic bias. They frequently predict that a formula rewrite will lead to a solvable state, failing to recognize when a transformation actually forces the proof into an irreversible dead end.

This failure to check intermediate states is exposed thoroughly by Chen et al. through the FineReason benchmark. When evaluating intermediate steps in complex logic puzzles, general-purpose models consistently fail to identify unsolvable intermediate states. Instead of taking corrective action or backtracking, the models confidently march down invalid logical paths, operating under the illusion that a correct answer remains reachable.

When applied to tool-use environments, this bias turns into hallucination. In the ToolBeHonest benchmark developed by Zhang et al., models were evaluated on scenarios where necessary tools were intentionally missing or restricted. Rather than diagnosing the task as unsolvable due to environmental constraints, top-tier models routinely made up non-existent tools, trying to force a solution at all costs.

The Financial Cost of Over-Optimism

This optimism isn't just an abstract logical flaw; it has concrete resource implications for autonomous agents. When an agent is deployed to execute multi-step web or coding tasks, it must monitor its remaining budget—whether measured in tokens, time, or API costs.

Two parallel studies on budget awareness—one by Lin et al. and another by the authors of Towards Budget-Aware Agents—demonstrate that LLM agents are remarkably poor accountants. Both papers show that models systematically underestimate the resources required to complete a task.

Crucially, this over-optimism prevents agents from identifying doomed trajectories early. An agent will continue spending tokens and executing actions on an impossible task, only realizing the failure at the very final stages of execution when its budget is entirely exhausted. This inability to detect early signs of failure results in massive, unnecessary resource waste.

Misattributing the Source of Failure

When an execution failure finally occurs, how does the model explain it? The systematic bias distorts the model's diagnostic capabilities, causing it to anchor incorrectly on the source of the error.

In evaluating benchmark defects, Park uncovers a phenomenon called "self-attribution anchoring." When an agent attempts to solve a coding task that contains an inherent, impossible defect in the benchmark itself, it does not realize the benchmark is broken. Instead, the model assumes the environment is perfect and repeatedly blames its own code for the failure. This internal anchoring drastically reduces the model's ability to objectively audit external infrastructure.

Conversely, this miscalibration can sometimes swing toward a pessimistic misattribution. Huang et al. observe that web agents can develop a pessimistic bias where they prematurely blame external system errors or task impossibility for their own execution failures. Because the agent misdiagnoses its own mistake as an unfixable external constraint, it abandons alternative navigation paths entirely, leaving tasks unresolved.

The Flip Side: Context Anxiety

While over-optimism dominates resource estimation during execution, the underlying miscalibration of task complexity can also manifest as preemptive panic. Igbinedion et al. document a phenomenon known as "context anxiety," where frontier reasoning models prematurely abandon complex tasks.

This occurs because the model systematically overestimates the number of tokens required to reach a solution, fearing it will exceed its token limits even when the task fits comfortably within its physical context window. Whether underestimating costs mid-execution or overestimating them at the start, the core issue remains the same: LLMs struggle to map their current state to an accurate resource trajectory.

Why the Bias Persists

The root cause of this systematic miscalibration lies in how language models are trained. Autoregressive pre-training exposes models to millions of tokens of human text, code repositories, and mathematical proofs. Crucially, this data overwhelmingly represents successful resolutions—completed code, solved proofs, and productive conversations.

Models rarely see traces of agents systematically tracking a dwindling budget, diagnosing an inherently broken task, or explicitly declaring an intermediate state as a dead end. Without explicit training on negative constraints and failure-mode diagnostics, language models default to the most statistically probable continuation: moving forward under the assumption that a solution is always just around the corner.

Key Foundational Work

ToolBeHonest: A Multi-level Hallucination Diagnostic Benchmark for Tool-Augmented Large Language Models (2024) — contributed 2 concept elements to this direction.
Mathematical Reasoning in Latent Space (2020) — contributed 2 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_systematic_optimism_bias_b55d8a2f,
  title = {systematic optimism bias},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/systematic-optimism-bias/rdr_b55d8a2fa88e4e26bd5b123550002e1d},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 1, frozen text):

@misc{lacuna_systematic_optimism_bias_b55d8a2f_v1,
  title = {systematic optimism bias},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/systematic-optimism-bias/rdr_b55d8a2fa88e4e26bd5b123550002e1d/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake