Finding the First Wrong Hop: Injected Error Cascades for Multimodal Reasoning Agents

When a vision-language agent answers a question like "What year was the company whose logo appears on this jersey founded?", it works in hops. It identifies the logo, searches for the company, reads the founding date, and answers. If the first hop misreads the logo, every later hop can be internally sound and the answer is still confidently wrong. We propose to teach a separate checking model, a step critic, to find where a chain first went wrong and what kind of mistake it was. We would train it on artificial cascades. We take a correct reasoning trace, deliberately corrupt one intermediate hop with an error the agent itself might plausibly make, and then let the agent keep reasoning from the corrupted hop so that the downstream damage is realistic. Every training trace then carries an exact label: the hop where the error started, its type (perception, retrieval, logical join, or unsupported claim), and which later hops merely inherited it. At inference, the critic tells the agent which single hop to redo and how, instead of resampling the whole chain.

The opportunity is concrete. Multi-hop multimodal benchmarks show that early visual misidentifications cascade into failures that grow worse and more overconfident with depth: Piecing It All Together: Verifying Multi-Hop Multimodal Claims reports roughly a 27% gap to humans on 4-hop claims and traces most failures to fine-grained visual misreadings. Meanwhile, the best multimodal multi-hop agent in our file, MMhops-R1, is trained only with outcome and format rewards and has no way to inspect its own intermediate hops. Synthetic error injection is the one tool that produces exact, cheap, step-level error labels. It has been used for single-image captions and for text-only math, but never for the intermediate hops of multimodal chains.

Core Claim

A critic trained on on-policy injected cascades will locate the origin hop of real, naturally occurring failures in multimodal multi-hop chains more accurately than a critic trained on Monte Carlo rollout labels. An "on-policy injected cascade" is a correct trace corrupted at one hop with an error drawn from the agent's own plausible alternatives, then continued by the agent itself. Using this critic to repair only the flagged hop will raise answer accuracy on MMhops and MMCV beyond what outcome-trained agents with Best-of-N reranking achieve at the same compute budget.

Two ingredients carry the claim. The first is that the critic predicts three states per hop: correct, origin error of a given type, or inherited error. No existing process reward model separates "this step caused the failure" from "this step faithfully used a bad premise." The second is that injections are drawn from the agent's own near-misses rather than random substitutions, which targets the synthetic-to-real gap directly.

Relation to Prior Work

Three works supply the motivation. MMCV documents the cascade and the depth-dependent overconfidence, but it applies its synthetic mutations (entity substitution, temporal edits) only to the final claim string to create REFUTE labels. We reuse those mutation operators but move them to intermediate hops. DocHop-QA finds that adding document images can degrade multi-hop answering, its "alignment tax," which suggests perceptual hops are a real failure point; it proposes no mitigation. Chain-of-Retrieval Augmented Generation shows in text that one bad retrieval step derails downstream reasoning. CoRAG builds training data by rejection sampling and keeps only correct chains. We start from the same pool of correct chains but use them as raw material for negatives.

Several works supply the method. REFINER perturbs intermediate reasoning steps with operator and number swaps to train a feedback model in text. Masked Summarization creates fluent but ungrounded content by hiding evidence from a generator. We use that idea for our "unsupported claim" injections. Hal-Eval and TLDR inject object, attribute, relation, counting and OCR errors into single-image captions to train hallucination detectors. Their error axes inform our perceptual injections, but neither models how an error propagates across hops or retrieval. Error Typing for Smarter Rewards (PathFinder-PRM) shows that classifying the error type before scoring a step improves text process reward models, reaching 67.7 PRMScore on PRMBench with 3x less data. We carry that decoupling into a multimodal setting with multimodal error types.

Other works constrain the design. VisualPRM is the closest competitor. It is a multimodal process reward model whose step labels come from the fraction of rollouts that still reach the right answer, and it is used for Best-of-N reranking on single-image math and logic. Rollout labels are cheap, but in deep chains they blame correct early hops for hard later hops. Separating that blame is exactly what our labels do. Learning from Synthetic Data Improves Multi-hop Reasoning warns that fine-tuning a policy on synthetic traces overfits to templates while reinforcement learning transfers. We therefore never fine-tune the policy on injected traces; only the critic sees them, as a discriminative task. Can Humans Dream of Electric Sheep? questions whether model-generated errors resemble organic ones. That is the main threat to this proposal, and it is why both the on-policy sampling of injections and the evaluation on human-labeled organic failures are central.

Research Program

Agent. We start with an agent in the MMhops-R1 style: a Qwen2.5-VL-7B backbone with image search, text search, and answer actions. We use released weights if available and otherwise reproduce the agent with its composite reward. From MMhops and MMCV training splits, we collect trajectories whose final answers are correct, as CoRAG does.

Injection. For each correct trajectory, we choose a hop and corrupt it with one of four operators.

  • Perceptual injections replace the agent's identified entity, attribute, or OCR string with the agent's own runner-up. This comes from its alternative decodes or from the second-ranked image-search hit, so the error is one this agent could actually make.
  • Retrieval injections swap in a passage about a near-namesake entity drawn from the same corpus.
  • Join injections apply REFINER- and MMCV-style rule edits to the reasoning text, such as swapping the relation, flipping a comparison, or shifting a date.
  • Unsupported-claim injections remove the supporting evidence from context and let the agent regenerate the hop, following the masked-summarization recipe.

The agent then continues from the corrupted hop. This step is essential. Without it, the corrupted hop contradicts the untouched original downstream text, and a critic can learn to spot the inconsistency rather than the error itself. Each trace is labeled per hop as correct, origin of type t, or inherited.

Critic. We fine-tune a 7B to 8B vision-language critic, sized to match VisualPRM-8B. It first predicts the error type and origin status for each hop, then a step score, in PathFinder-PRM's two-pass style.

Repair at inference. The agent runs. When the critic flags an origin hop k with type t, the agent re-executes only hop k with a type-specific action and then regenerates the downstream hops:

  • for a perceptual error, it re-queries image search on a crop or takes the next candidate;
  • for a retrieval error, it re-searches with the flagged document excluded;
  • for a join or unsupported-claim error, it resamples the reasoning for that hop.

The number of repair rounds is capped so that total tool calls and generated tokens match a Best-of-N budget.

Optional second stage. Only if the first stage succeeds, we test the critic's origin-hop signal as a process reward during the agent's reinforcement learning. This lets us ask whether step supervision from synthetic cascades helps where Learning from Synthetic Data saw outcome-only reinforcement learning succeed.

Human-labeled test set. We build one new evaluation resource: about 1,000 organic failed trajectories from the base agent on MMhops and MMCV test splits. Annotators mark the first wrong hop and its type. Inter-annotator agreement on origin hop is reported, and if it is low we treat that as a finding about the task itself.

Baselines

The naive reference point is the base agent with greedy decoding and no critic. Beside it we run the agent with self-consistency majority voting at matched sample count. This tells us how much of any gain comes simply from spending more compute. MMhops-R1 itself reports 51.35% accuracy on MMhops Bridging and 22.01% on Comparison, against 36.62% and 8.76% for GPT-4o and 46.58% on Bridging for Gemini-2.5-flash. Our reproduction must land near 51.35% Bridging before any comparison is meaningful.

The strongest existing competitor is the same agent with Best-of-N reranking by a VisualPRM-style critic. This critic has the same architecture and size as ours and is trained on Monte Carlo rollout labels computed over the same pool of agent trajectories, with the same number of training examples and the same tuning budget. It is the strongest competitor because VisualPRM is the published multimodal process reward model, and it reports 5.9- and 8.0-point gains as a Best-of-N verifier on InternVL2.5-78B and MiniCPM-V2.6-8B. No reported result found on Lacuna for VisualPRM on MMhops or MMCV, so we produce that number ourselves. Beating it means two things. First, higher accuracy on MMhops Bridging and Comparison at every matched budget of N in {4, 8, 16} tool-call-equivalents. Second, higher origin-hop localization accuracy on the human-labeled organic failure set.

For claim verification, the reference is Gemini 1.5 Flash's 70.92% average F1 in MMCV's open-book setting, the best reported model, against human experts at 80–90%. Here the target is narrowing the 4-hop gap specifically, not just the average.

A prompted frontier judge serves as a further baseline: a large proprietary vision-language model asked to locate and type the first wrong hop, with the same repair procedure.

Ablations isolate the key ingredients:

  • Injection without downstream continuation (corrupt hop k, keep the original downstream text) tests whether realistic cascades matter.
  • Random rather than on-policy corruptions tests whether matching the agent's own error distribution matters.
  • A critic with only correct/incorrect labels (no origin-versus-inherited distinction and no typing) tests the three-state output.
  • Full resampling on a flag instead of single-hop repair tests whether localization, rather than detection alone, drives the gain.
  • A mixture critic trained on rollout labels plus injected labels tests whether the two label sources complement each other.

Why not just…?

Why not just use Monte Carlo rollout labels, as VisualPRM does? Rollout labels need no injection machinery and come from real errors, so this is a serious alternative. In deep chains, though, the fraction of successful continuations from a hop mixes "this hop is wrong" with "the remaining hops are hard." A correct 1st hop of a 4-hop chain can score low, and an inherited-error hop looks the same as the origin. Our labels are exact by construction. The head-to-head comparison on human-labeled origin hops settles this. If the rollout-trained critic localizes as well as ours, the injection pipeline adds nothing and we would recommend the mixture or rollout labels instead.

Why not just train the agent harder with outcome reinforcement learning and spend more compute on Best-of-N? CoRAG shows that test-time search scales log-linearly in text, so a bigger N might close the gap. This is a real threat. But Best-of-N can only pick among chains, and when early perceptual hops fail systematically, as MMCV reports for logos and fine visual detail, many samples repeat the same misreading. Targeted repair re-executes the faulty hop with a different action. We plot accuracy against matched compute for both. If Best-of-N curves meet ours at modest N, the critic's value is limited to diagnosis rather than accuracy.

Why not just ask a frontier model to find the wrong step? Prior work finds that vision-language judges themselves hallucinate errors and misread layouts (Chen et al.; Kaiyi Huang). Hal-Eval's trained evaluator reaches only 47.34% Pearson correlation with humans even on single-image captions. GPT-4o as an agent reaches 36.62% on MMhops Bridging, which suggests it would also struggle to verify those chains. We still run the prompted judge on the human-labeled set, because a strong result there would make a trained critic unnecessary.

Why not just apply an anti-hallucination decoding method at each perceptual hop? RITUAL combines predictions on original and randomly transformed images and lifts POPE accuracy on LLaVA-1.5 from 84.13% to 88.87%. IRIS scores responses by how much less probable they become when the image is removed, and cuts sentence-level hallucination on Object-HalBench from 54.70 to 8.66. Both target perceptual errors only. They cannot fix a wrong retrieval or a flipped comparison, and IRIS reports only modest gains on compositional reasoning. We include RITUAL applied at every image hop as a baseline, with its doubled per-hop cost counted in the compute budget. The per-type breakdown on the organic set shows what share of failures it could possibly address.

Decisive Checks

The first and most important check is transfer to organic errors. On the human-labeled failure set, we measure origin-hop accuracy and type accuracy for our critic, the rollout-trained critic, and the frontier judge. If our critic does not beat the rollout critic here, the claim fails regardless of downstream accuracy. We also report recall per error type, to see whether injected perceptual errors transfer better or worse than injected join errors.

The second check is end-task accuracy at matched compute on MMhops Bridging and Comparison and on MMCV broken down by hop count. INFOSEEK, where MMhops-R1 reports 33.2%, serves as a single-hop check that repair does not hurt easy questions. MultiModalQA cross-modal questions, where ImplicitDecomp reports 51.7 F1, serve as a held-out transfer set never used for injection.

The third check is calibration by depth. Using the critic's origin-error probability as an abstention signal, we measure calibration error separately for 1- through 4-hop MMCV claims. The test is whether the overconfidence-with-depth pattern MMCV reports shrinks.

The fourth check is that the critic remains a competent general step checker. We evaluate it on VisualProcessBench, which has human step labels, so that we can tell whether gains come from multi-hop structure or from overfitting to our injection templates.