TL;DR
While hallucinations in natural language generation produce incorrect text, hallucinations in LLM agents produce real, potentially dangerous actions—like leaking credentials or submitting broken code. MIRAGE-Bench introduces the first unified benchmark to categorize, elicit, and evaluate agentic hallucinations across diverse interactive environments. By freezing decision points into reproducible "contextual snapshots" and evaluating them via a risk-aware LLM-as-a-Judge, the authors show that agent hallucinations are pervasive across both open-source and frontier proprietary models.
What Problem Does This Paper Address?
Evaluating hallucinations in interactive agents is far more challenging than in standard single-turn QA:
- Context-dependent actions: An agent's action (e.g., clicking a button) cannot be verified with standard fact-checking tools; its correctness depends on instructions, environment observations, and past steps.
- Environmental fragmentation: Agents operate across disparate platforms (browsers, operating systems, code repositories), making universal evaluation environments cumbersome and expensive to maintain.
- Reproducibility issues: Stochastic environment transitions and branching paths make failure modes unpredictable and difficult to replicate.
The authors address this gap by creating a unified testbed that isolates and measures unfaithful agent behavior in controlled, reproducible decision points.
Methods
The authors structure their benchmark around three key pillars:
- Three-Part Hallucination Taxonomy: Agent hallucinations are classified by what part of the context they contradict:
- Task Instruction Unfaithfulness: Violating goals, assuming unstated user intents, or hallucinating unsupported parameters.
- Interaction History Unfaithfulness: Repeating failed actions or ignoring past failure feedback.
- Environment Observation Unfaithfulness: Interacting with non-existent UI elements or assuming state transitions occurred when they did not.
- Contextual Snapshot Strategy: Instead of conducting expensive, open-ended environment rollouts, the benchmark freezes the agent’s exact state (instruction, history, and observation) right before a failure-prone decision. To scale test cases efficiently, they use automated contextual editing (e.g., using
o4-minito inject diverse out-of-scope user queries into accessibility trees while preserving over 99% of DOM structure). - Risk-Aware LLM-as-a-Judge: Using a zero-shot, two-step rubric, an LLM judge evaluates the agent's internal reasoning (thought) and generated action. Decisions are scored into three categories: Faithful (Utility = 1.0), Incomplete/Exploratory (Utility = 0.5), or Hallucinated (Utility = 0.0).
- Subsequence Association Trace (SAT) Attribution: The authors use token/sentence-level causal analysis to trace why agents hallucinate, measuring which parts of the prompt drive the model's unfaithful generation.
Datasets
Rather than starting from scratch, the authors audit failure trajectories across six established agent benchmarks to instantiate six critical real-world risk settings:
- Out of Scope Queries (OSQ) – derived from TheAgentCompany and $\tau$-Bench.
- Unexpected Environmental Transitions (UET) – from TheAgentCompany and WebArena.
- Unachievable Goal States (UGS) – from WorkArena, WebArena, and OSWorld.
- Ill-specified Instructions (ISI) – from SWE-Bench and WebArena.
- Flawed Interaction History (FIH: Repetitive & Erroneous) – from SWE-Bench, WebArena, and WorkArena.
- Pop-up Distractions (PUD) – from WebArena and OSWorld.
Key Insights and Ideas
- Cognitive Inertia: Agents frequently stick to a preconceived plan even after an action fails or the environment doesn't update. Instead of reacting to real-time feedback, they assume the prior step succeeded.
- Training Bias Toward "Happy Paths": Attribution analysis reveals that hallucinated actions are heavily driven by subsequences representing standard, successful workflows (e.g., the last action taken and the button for the intended next step). Because imitation learning and RL datasets predominantly consist of successful trajectories, models receive weak gradient signals for recovering from rare error messages.
- Scale Does Not Solve Agent Hallucination: Frontier proprietary models hold only a slight advantage over open-source models in hallucination rates. In prefix-steered introspection tests,
GPT-4owas actually more prone thanGPT-4o-minito overwrite explicit error feedback with assumed success.
Key Results
- Pervasive Failures Across Frontier Models: Across twelve evaluated LLMs, overall hallucination rates (HR) remained alarmingly high:
- Proprietary: Claude-3.5-Sonnet (HR 30.8%, Utility Score 0.589); GPT-4o (HR 33.9%, Utility Score 0.569); Gemini-2.5-flash (HR 30.8%, Utility Score 0.586).
- Open-source: Qwen2.5-72B-Instruct (HR 33.4%, Utility Score 0.579); Llama-3.3-70B-Instruct (HR 43.3%, Utility Score 0.488).
- Hardest Risk Settings: Agents performed worst on Unachievable Goal States (HR often exceeding 40–60%) and Unexpected Environmental Transitions (HR often 30–55%).
- Judge Reliability: LLM judges demonstrated strong agreement with human evaluators (overall Accuracy > 0.75), with Claude-3.5-Sonnet achieving a 0.895 ZeroAcc (accuracy on detecting hallucinated actions).
Limitations and Future Work
- Static vs. Multi-turn Evaluation: Freezing states into snapshots prevents measuring whether an agent that takes a hesitant or exploratory step (scored as "Incomplete") would eventually self-correct in subsequent turns.
- Manual Curation Bottlenecks: Certain risk settings (such as determining the exact moment an infeasible field appears in enterprise workflows) still require manual filtering.
- Implications for Agent Training: The authors conclude that simple prompt engineering or behavior cloning on optimal paths is insufficient; agent developers must intentionally train models on error-injection curricula, failure-recovery episodes, and negative trajectories.
Summary updated