Do Vision-Language Planners Talk Themselves Out of What They See?

Vision-language models (VLMs), which are language models that also read images, write household plans that sound sensible and still fail when carried out. A typical failure is "take the milk out of the fridge" while the fridge is visibly closed. A common explanation is weak perception. We propose a second explanation that can be tested directly: the model's own half-written plan acts as misleading text that overrides what the image shows. Static image question-answering work already shows that wrong text in the prompt makes these models ignore visual evidence. A plan prefix is exactly that kind of text, and the model wrote it itself. If this is right, a cheap fix follows. Before each step runs, convert its physical preconditions into short yes/no questions about the image, and have the same model answer them with the plan removed from its context.

The note pairs a measurement with a method. The measurement is the "self-overrule gap": how much worse a VLM answers a question about scene state when its own plan is in context than when the plan is absent. The method is a three-tier audit run during planning. Tier one asks about perception with the plan context stripped. Tier two checks whether all of a step's preconditions can be satisfied, and stops if they cannot, so the model reports that the task is unsolvable instead of inventing a way around a missing object. Tier three accepts the step. The tiered structure is the hierarchical task taxonomy idea, here used to decide what happens during planning rather than only to score failures after the fact.

Core Claim

A large share of physical precondition errors in VLM procedural planning comes from the model's text conditioning overriding its vision, not from an inability to see. The model can usually answer "is the fridge closed?" correctly from the image alone. It answers incorrectly once its earlier steps have implied the fridge is open. We define the self-overrule gap for a model $M$ as

$$\Delta(M) = \Pr[\,M(q, I) \text{ correct}\,] - \Pr[\,M(q, I, p_{<t}) \text{ correct}\,],$$

Here $q$ is a yes/no question about a precondition of step $t$, $I$ is the current image, and $p_{<t}$ is the plan the model has written so far. Ground truth comes from the simulator state. Two predictions follow. First, $\Delta$ is clearly positive and grows with plan length, which would match the collapse at long horizons. Second, auditing with the plan stripped recovers most of the gap. Auditing with the plan kept, which is ordinary self-verification, recovers little or makes things worse.

Where This Sits in Prior Work

Three findings motivate the proposal. PhD builds a static visual question-answering benchmark with an "incorrect context" mode and shows the overrule effect. LLaVA-1.5-L drops from 0.443 to 0.011 when the prompt text contradicts the image, and larger language backbones can ground worse because their priors are stronger. PhD never looks at sequential plans, where the misleading context is produced by the model. Work on autoregressive compounding (Holt et al.) argues that models treat their own earlier outputs as established fact. Our proposal joins these two findings and measures the result in embodied plans. ActPlan-1K shows that VLM plan correctness is low: Gemini-Pro-1.5 reaches 32.0% on normal activities and 22.8% on counterfactual ones, whose plans are also longer (31.93 versus 23.95 steps). That is the setting where we expect overrule to be strongest.

Two lines of work supply the diagnostic structure we reuse. Embodied Agent Interface splits embodied decision making into goal interpretation, subgoal decomposition, action sequencing and transition modeling. It finds that 45.2% of errors are trajectory-feasibility (precondition) violations. However, it works purely on ground-truth symbolic scene graphs, so it cannot tell a perception failure from an overrule failure. We take its precondition annotations and place images in front of the model. ToolBeHonest supplies the tiered solvability framing: detect whether a task is solvable, find the bottleneck subgoal, describe what is missing. It also reports a large drop on unsolvable tasks, where GPT-4o's overall score is 37.0. We carry this framing from software tools over to missing physical objects and affordances. In both papers the taxonomy only scores models. In ours it gates execution.

Two papers constrain the design. Open-loop VLM Robot Planning finds that self-verification lowers EgoPlan-Bench performance by more than 10% for fine-tuned models and that self-consistency also hurts. A naive "check your plan" audit is therefore ruled out, and our design has to explain why it differs. Context isolation is our proposed explanation. Explainable Procedural Mistake Detection describes a "complexity trap": its question generator learned to ask finer-grained questions than the vision encoder could answer. So we restrict tier-one questions to coarse object-state predicates such as open/closed, inside, on top of, and present/absent, and we measure answerability directly.

The neighboring systems differ along a clear axis. HP-KG retrieves text-only procedural knowledge, which fixes missing procedural knowledge (forgetting to open the kettle lid) but not misreading the scene. Critic-V trains a separate critic VLM for static reasoning questions. Sentinel-VLA monitors low-level action execution to trigger recovery. PAMAR audits plans in 2D user interfaces and fills gaps with web search. None of them asks whether the planner's own context corrupts its perception, and none separates the perception check from that context.

Research Program

The first stage is a measurement study. We use ActPlan-1K's iGibson/BDDL instances (BDDL is the formal language those tasks use to specify object states), together with the BEHAVIOR split of Embodied Agent Interface rendered in OmniGibson from BEHAVIOR-1K. From these we build a probe set. Each probe consists of a scene image, a model-generated plan prefix, and coarse precondition questions whose answers are read from the simulator state. We record each model's answer in three conditions: no context, with its own plan prefix, and with an injected false prefix in the style of PhD's incorrect-context mode. We then report $\Delta$ by predicate type, by step depth, and by model size. If the PhD size paradox carries over, larger backbones should show larger gaps. This stage also yields a per-step attribution of every precondition violation. A violation is "perception" if the model answers wrongly even without context. It is "overrule" if the model answers correctly without context but wrongly with it. It is "knowledge" if no precondition question was violated but the step was still invalid.

The second stage builds the in-the-loop audit. The planner proposes step $t$ along with its preconditions, written as predicates in the BDDL vocabulary. Each precondition becomes a templated yes/no question. The same VLM answers it with only the image and the question, a few tokens and no plan. If every precondition holds, the step is accepted. If one fails, the planner is told which fact failed, phrased as an observation, and it rewrites the step. If the failed precondition cannot be fixed by any object present in the scene, the system stops and reports the bottleneck, mirroring ToolBeHonest's levels. Templated questions keep the audit cheap and out of the complexity trap. The audit runs only on steps that have preconditions, not on every frame, which addresses the per-step cost concern that Sentinel-VLA raises.

The third stage adds an unsolvable split. For a subset of ActPlan-1K and EAI-BEHAVIOR instances, we remove or disable an object the task requires, for example by deleting the knife or locking the cabinet. This creates physical analogs of ToolBeHonest's missing-tool and limited-functionality scenarios. Here the right behavior is to abstain and name the bottleneck.

Baselines

The naive reference point is the unaugmented VLM planner prompted to output a step sequence from the image and instruction. We run it with the same backbones as the audit: an open model such as Qwen2-VL-7B, which VLABench uses, plus one proprietary model. The strongest published number on the main benchmark is Gemini-Pro-1.5 on ActPlan-1K, with 32.0% correctness on normal activities and 22.8% on counterfactual ones (ActPlan-1K). We rerun that configuration under our protocol rather than relying only on the quoted figure. Beating this baseline means a gain in correctness on counterfactual activities without a loss in commonsense satisfaction.

The ablation that isolates the key ingredient is a context-retained audit. It is identical to our method, with the same templated questions, the same rewrite loop and the same number of calls, except that the question is answered with the plan prefix still in context. This is self-verification in the sense of Open-loop VLM Robot Planning, made as strong as we can make it. The difference between the two arms is a direct estimate of how much the gain comes from isolation. If the two perform the same, our core claim is false even if both beat the naive planner.

The strongest existing method for knowledge-side errors is HP-KG retrieval (HP-KG). It reports a 17.64% improvement in planning for 7B models on ActPlan-1K and is the reason a 7B model can beat a 72B one there. We use its released graph and retrieval settings with the same backbone, and we also test HP-KG combined with our audit. We expect the two to be largely complementary, since one supplies procedural knowledge and the other checks the scene. Beating HP-KG means higher correctness on counterfactual instances, where retrieved procedures conflict with the altered scene. On normal instances we expect at least parity when combined with it.

A perceive-first symbolic baseline has the VLM write a full scene-state description once, as BDDL predicates. A symbolic validator then checks each step's preconditions against that description. This is the cleanest competitor because it also separates perception from planning. We give it the same predicate vocabulary and allow at least as many tokens as our audit uses in total.

A flat critic baseline prompts the same VLM to criticize the whole plan given the image, in the style of Critic-V. Because no embodied Critic-V checkpoint is described in the file, it is prompted rather than trained, and it is given the same budget of rewrite rounds. Every arm uses the same instances, backbones, decoding settings, and a matched prompt-tuning budget on a shared development split. We report model calls and tokens per plan so that compute is visible.

Why not just…?

Why not just use a bigger model? Scale works against the mechanism we are measuring. PhD reports that larger language backbones can ground worse because their priors are stronger, and even a frontier model reaches only 22.8% correctness on counterfactual ActPlan-1K activities. The measurement stage reports $\Delta$ across model sizes. If the gap shrinks to zero at the largest scale we test, this alternative wins at that scale, and we will report it as a result.

Why not just ask the model to double-check its plan? That is self-verification, and Open-loop VLM Robot Planning reports that it costs more than 10% on fine-tuned models. Our hypothesis explains why it fails: the check is made inside the same context that caused the error. The context-retained ablation is exactly this alternative with matched compute, so the comparison settles the question directly.

Why not just extract a scene graph once and plan symbolically? This is the real threat, and it may match our method on short, static tasks. We expect it to lose for two reasons. Writing out every predicate makes the model answer many questions nobody needs, and errors accumulate across them. A snapshot taken once also goes stale as the plan changes the world, whereas targeted questions are asked when each step is about to run. If the perceive-first baseline equals our audit at similar cost, the method contribution shrinks to "separate perception from plan context". The overrule measurement and the physical unsolvability split would remain the contribution, and we would say so.

Why not just add procedural knowledge, as HP-KG does? Retrieval fixes steps the model does not know. It does nothing about a model that knows to open the fridge but believes the fridge is already open. The error attribution from the measurement stage tells us what fraction of violations each approach can address, and the combined arm tests whether the two add up.

Proposed Experiments and Decisive Checks

The first decisive check comes before any system is built. On the probe set, is $\Delta$ clearly above zero, measured with simulator ground truth, and does it grow with step depth? If it is near zero, failures are pure perception, isolation cannot help, and the project becomes a perception study. The injected-false-prefix condition gives a dose-response check: a false prefix should widen the gap beyond what the model's own prefix causes, as PhD's incorrect-context mode does in the static case.

The main experiment compares all arms on ActPlan-1K normal and counterfactual splits and on rendered EAI-BEHAVIOR. It reports plan correctness and commonsense satisfaction as ActPlan-1K defines them, the rate of precondition violations, and EAI's missing-step and extra-step counts. Success means that the context-isolated audit beats the context-retained audit by a clear margin on counterfactual instances and beats or complements HP-KG. On the unsolvable split, we report abstention precision and recall and bottleneck identification accuracy, following ToolBeHonest's tier scoring. The naive planner should almost never abstain, given ToolBeHonest's over-optimism finding. For transfer to real images, we apply the same audit to EgoPlan-Bench multiple-choice planning, where the prior negative result for self-verification gives a sharp test. We also report answerability of tier-one questions by predicate type, to check that the complexity trap was avoided. Finally, we report cost per plan, because an audit that improves correctness but needs as many calls as checking every frame would not be usable on a robot.