Imagined Consequences as Evidence for Embodied Safety Critics

Language-model agents that control robots often know that an action is dangerous and do it anyway. Asked in the abstract, they can say that a metal bowl does not belong in a microwave. Asked to carry out a household task, they put it there. This proposal tests a simple fix. Before the robot acts, a separate image-editing model draws what the scene will look like after the action. A safety critic then judges that predicted picture of the outcome, not the action's description. If it sees sparks, a broken glass, or water on a phone, it rejects the step, writes the reason into a running log of rejected steps, and the planner tries again. The bet is that models recognize hazardous states far more reliably than hazardous actions. Showing the critic the consequence should therefore close part of the gap between knowing and doing.

Prior work motivates this from two sides. LM Agents May Fail to Act on Their Own Risk Knowledge finds models above 98% on abstract risk questions yet only 4.8%–25.6% safe on execution. Its fix is a verifier paired with an "abstractor" that rewrites a trajectory as a high-level summary, which raised GPT-4o-mini's safe pass rate from 27% to 71% on ToolEmu. SafeAgentBench shows the same gap in a simulated house: the best baseline, ReAct with GPT-4, rejected only 10% of detailed hazardous tasks. Our proposal extends the abstractor idea. Where that work changes how the action is described, we show the critic what the action produces. Our central experiment asks whether showing the outcome as a picture adds anything over describing it in words.

Core Claim

On embodied tasks where the hazard comes from how objects interact, a critic that inspects an imagined after-state image will catch more hazards than three alternatives at matched cost: a critic that inspects the action text, a critic that inspects an abstractor summary, and a critic that inspects a text prediction of the after-state. The gain should concentrate on what we call context-contingent hazards. These are cases where the instruction is benign and the danger depends on scene details the instruction never mentions, such as a fork already inside the microwave or a filled cup above a laptop.

The four input directions play different roles. Iterative plan refinement supplies the core loop: propose, evaluate, correct. Reasoning-guided image generation supplies the means of evaluation, a predicted after-state. Physical safety grounding is the setting where the method is tested. Multi-agent task decomposition is the design constraint: the roles are kept in separate models so the critic never grades its own plan.

Research Program

The system has three roles.

The Planner is an LLM agent producing SafeAgentBench's high-level actions, such as slice, break, and fillLiquid.

The Imaginer is an instruction-based image editor. Given the current egocentric frame and one proposed action, it renders the predicted next frame. We start from Bagel and follow the recipe of WorldEdit, which trains physical chain-of-thought before rendering with a causal-verification reward. That reward asks an external model to recover the instruction from the generated image. WorldEdit's own data covers material changes such as breaking and heating. We add before/after frame pairs rendered by executing actions in AI2-THOR. The simulator gives these pairs exact ground truth, in the same spirit as the executable-code pairs in Factuality Matters. These scenes are kept disjoint from the evaluation scenes. To limit the scene-composition drift reported by AlphaGRPO, we add a penalty on pixel changes outside the region of the object being acted on.

The Inspector is a vision-language model. It receives the before and after images plus a short safety constitution in the style of TrustAgent, and outputs accept or reject with a stated reason.

When the Inspector rejects a step, the action and the reason are appended to a correction trace, a running log of rejected steps. The Planner then resamples conditioned on that trace. This is the mechanism of CorrectionPlanner, lifted from driving motion tokens to symbolic household actions with image evidence. If the Planner revises the same step three times without acceptance, the task is refused.

Two design choices respond directly to prior work.

Not every step is imagined. CoT-VLA reports a 7x slowdown from generating a subgoal image at every step. AdaNav found that triggering reasoning only when uncertain fired on 2.5 of 55 steps. Following AdaNav, we imagine a step only when it changes an object's state, or when a cheap text-only Inspector pass is uncertain. Navigation steps are passed through.

The Imaginer is kept separate from the Planner. CoT-VLA uses one model to draw a goal image and then act on it, open-loop. That model has every incentive to draw the outcome it intends. Our Imaginer is asked only "what happens if," and is trained on simulator ground truth.

To test the context-contingent claim, we split SafeAgentBench's 450 hazardous tasks into two groups. One group has hazards stated or implied by the instruction text. The other has hazards that depend on scene state. Two annotators label each task, and disagreements are resolved before any system is run.

Baselines

The naive reference is ReAct with GPT-4 run directly on SafeAgentBench. Its reported result is 10% rejection of detailed hazardous tasks, with risk rates up to 69% for some frameworks (SafeAgentBench). We re-run it in our harness so that all comparisons share one Planner.

The strongest existing competitor is the verifier-abstractor system from LM Agents May Fail to Act on Their Own Risk Knowledge. It reports 71% safe pass for GPT-4o-mini on ToolEmu. That number comes from a non-embodied benchmark; we found no reported result for it on SafeAgentBench on Lacuna. This is the most direct competitor because it already separates verification from generation and already reframes actions before judging them. The only thing it lacks is our hypothesized ingredient.

We also run a TrustAgent-style post-planning inspector. No quantitative result for it was found on Lacuna. As a cheap front-end filter we run Pinpoint (Advancing Embodied Agent Security), which reports 94.58% accuracy on explicit threats.

Fairness rules apply to all systems. Every critic uses the same Inspector backbone, the same constitution, the same correction-trace loop, and the same three-revision cap. Every system gets the same budget of Inspector calls per task, and prompts are tuned on the same held-out scenes.

Two references bracket our method. An oracle variant executes the action in a copied simulator and shows the Inspector the true rendered after-frame. This sets the ceiling for imagination and separates Imaginer error from Inspector error. The key ablation replaces the Imaginer with an LLM that writes a text prediction of the after-state.

Beating the verifier-abstractor means two things together. On the context-contingent split, our rejection rate must be significantly higher, by paired bootstrap over tasks. On the 300 safe control tasks, our completion rate must be no worse by more than 3 points.

Why not just…?

Why not just ask the LLM to describe the consequence in words? This is the real threat. If a text after-state prediction matches the image on the context-contingent split, the image generator is unnecessary, and we will report that. A plain textual consequence predictor would then be the contribution. We expect images to win only where the hazard depends on spatial or visual detail that the scene's object list underspecifies, such as containment, proximity, or liquid paths. The per-category breakdown will show where any gap lives.

Why not just give a text critic the full symbolic scene state from the simulator? In AI2-THOR this metadata exists, and a text critic with it may perform well. It is not available on a real robot, however, where perception gives frames rather than ground-truth object lists. We therefore run this as a second privileged reference and evaluate our method with image input only. If the image Inspector approaches the privileged text critic, the image route is a viable stand-in for missing metadata.

Why not just filter instructions at the input? Pinpoint is fast and accurate on explicit threats. Many SafeAgentBench hazards, however, are benign phrasings like "put the bowl in the microwave," where nothing in the text is malicious. We report Pinpoint alone and Pinpoint stacked in front of our loop. Our expectation is that the stack helps on the text-stated split and adds little on the context-contingent one.

Why not just a single model that draws a goal image and acts, as in CoT-VLA? Its goal image depicts what the policy intends, which is open-loop with no rejection path. We test this concern directly with a self-imagination ablation, in which the Planner itself draws the after-state. We expect it to rubber-stamp its own plans more often than the separate Imaginer does.

Proposed Experiments

The main experiment runs all systems on SafeAgentBench's 750 tasks, scored by its rule-based execution evaluator and GPT-4o semantic evaluator. We report four measures: rejection rate on hazardous tasks, risk rate, completion on safe controls (to expose over-correction, which CorrectionPlanner flags as a failure mode), and the number of Imaginer calls and wall-clock time per task.

Three checks are decisive.

The first is the image-versus-text-consequence gap on the context-contingent split. This determines whether the idea survives.

The second is the gap between the oracle and our method. If it is large, the bottleneck is image quality, and we score the Imaginer separately on held-out AI2-THOR transitions and WorldEdit-Test, where WorldEdit reports 4.07.

The third is the adaptive trigger. It must recover most of the benefit of imagining every step at a fraction of the calls. Otherwise the latency cost argued against by CorrectionPlanner stands.

A small secondary study uses EAsafetyBench-Drone instructions. It checks whether consequence images help for hazards involving people, where a predicted frame may be least reliable.