Two Readers Before Every Hop: Verified, Anchored Reading for Multi-Hop Document Reasoning

When a model answers a question that spans several charts, tables, or pages, it reads one value, uses that value to decide what to look at next, and repeats. A single misread number or a value taken from the wrong bar at the first step sends every later step in the wrong direction, and nothing in current systems catches it. We propose to make every intermediate read verifiable. At each hop, two readers that work differently look at the same image region. One is the vision-language model, which reads from pixels. The other is a separate OCR model (optical character recognition: a model that transcribes text in an image into characters). The system checks that their readings agree, and that the label next to the value names the entity the reasoning is looking for. Only values that pass this check enter the model's working memory. Each one is stored with its page and bounding box. When the readers disagree, the system zooms in and re-reads that region, rather than reasoning onward from a guess.

The opportunity is that the two research lines have each solved half the problem. Iterative retrieval shows that small models reasoning step by step can beat much larger one-shot systems. Explicit OCR shows that a compact transcription model can out-read frontier vision models on dense text. Neither line handles the failure that dominates document benchmarks today, which is a wrong or misattributed intermediate value. On the synthetic single-page benchmark DocHop, accuracy falls from about 85% on 2-chart pages to about 40% on 6-chart pages. The paper attributes the losses mainly to "entity mismatch": the model understands the rule but reads the value from the wrong series.

Core Claim

Intermediate-hop errors in multi-hop document question answering are mostly perception and binding errors, not planning errors. They can be detected cheaply, because a pixel-based reader and an independently trained OCR reader rarely make the same mistake on the same region. The proposal makes this concrete with three pieces:

  • Evidence ledger. This is a small structured memory. Each entry holds a named variable, its value, the page and bounding box it came from, both readings, and the label text found next to it.
  • Commit rule. A value is committed to the ledger only when two conditions hold. The two readings agree after normalizing numbers and strings, and the OCR'd row, legend, or axis label matches the entity named by the reasoning so far.
  • Targeted re-reads. Disagreement triggers re-reading of that one region, not of the whole page.

We predict two things. Most of the gain will come on dense, many-visual pages and on chain-style questions. And the gain should not be reproducible by spending the same compute on unconditional zooming or on repeated sampling from a single reader.

Relation to Prior Work

Several papers motivate this proposal by documenting the gap.

  • Reasoning-OCR builds 150 single-image questions in which OCR cues serve as premises for logic. Models above 95% on DocVQA fall below 50% at open-source 7B–13B scale. Text-centric OCR-tuned models underperform generic ones. This constrains our design: we keep the OCR model separate rather than fine-tuning the reasoner on transcription.
  • DocHop-QA benchmarks multi-document PubMed questions. It reports that adding page images hurt relative to text-only input, which the authors call an "alignment tax." This is why our ledger feeds the reasoner only the few anchored values it asked for, not raw OCR dumps or every page image.
  • M^3-Bench names visual grounding the "first-mile problem" of tool chains, but offers no repair mechanism.

The closest systems all iterate, but none verifies intermediate reads.

  • ViDoRAG uses a seeker, an inspector that re-reads candidate pages at high resolution, and an answer agent that checks global consistency. Its consistency check operates on drafted answers at the end, and its inspection is unconditional and page-level. Ours is triggered by reader disagreement and region-level.
  • $G^2$-Reader builds a content graph over paragraphs, tables, and figures, plus a planning graph of sub-questions. It reaches 66.21% average on VisDoMBench. Its nodes stop at the figure or table level, it acknowledges fine-grained visual text as a weakness, and it has no on-demand re-reading.
  • LLM-Guided Planning over nuclear regulatory documents keeps a typed memory graph and an early-stopping goal test (85.5% accuracy on early stops versus 78.3% on full-budget runs). It calls vision only at the last step, so intermediate hops cannot read figures.
  • MMhops-R1 trains a 7B agent with reinforcement learning to call image and text search. It reaches 51.35% on Bridging but only 22.01% on Comparison, and it has no OCR tool.

The method components come from three sources.

  • From IRCoT and ITER-RETGEN we take the loop that interleaves reasoning steps with retrieval. ITER-RETGEN's finding that gains converge in about two iterations sets our default hop budget.
  • From General OCR Theory we take GOT, a 580M end-to-end OCR model, as the second reader. It supports region-level reading and reaches 0.640 AP@high on PlotQA versus 0.223 for GPT-4V.
  • The coordinate-grounded reading of AskChart and the earlier OCR-plus-position fusion of LoRRA and LaRA show that explicit text tokens help. All of them inject OCR once, statically, for a single image.

The difference from all of this work is that OCR here acts as a verifier inside the loop, not as an input stream.

Research Program

The system uses one open vision-language model as planner and pixel reader: Qwen2.5-VL-7B, and Qwen3-VL-32B for comparison with $G^2$-Reader. It uses GOT as the OCR reader and a fixed page retriever shared by all compared systems. Each hop runs in five steps:

  1. The planner states the next variable it needs, for example "Q3 revenue of the entity labelled Certified Prudent Operator," and proposes a page and a box.
  2. The vision-language model reads the value from the crop.
  3. GOT transcribes the same crop plus a margin. The margin captures the nearest row header, legend entry, or axis tick.
  4. A deterministic checker compares the two values after normalization. It also fuzzily matches the OCR'd label against the entity string held in the ledger.
  5. On agreement the entry commits. On disagreement the system re-reads at up to two higher zoom levels, then expands the box to include the full legend or table header. If the readings still disagree, it stores both candidates as an unresolved entry, and the planner must either seek corroborating evidence elsewhere or answer with that uncertainty exposed.

The system stops when every variable in the current plan is committed. A hop cap acts only as a backstop, which addresses the over-refinement drift that $G^2$-Reader reports beyond three iterations.

The project has three analysis threads.

  • Error independence. We measure how often the two readers err on the same region, using ground-truth chart data from DocHop. Low correlated error is the premise of the whole method.
  • Cascade curve. We inject controlled errors into the first committed value and trace how final accuracy decays with and without the commit rule.
  • Error taxonomy. We sort remaining failures into misread digit, wrong series, wrong page, and wrong plan, so we can say which ones verification fixes.

Baselines

The naive reference point is the same vision-language model given the question and retrieved page images in a single pass, with no loop. On DocHop, the file reports 7–22% for open-source 7B–38B models and 60.18% for GPT-5.2 Reasoning (DocHop). A method that does not clearly beat the single-pass model under the same backbone has failed.

The strongest structured competitor is $G^2$-Reader. It reports 66.21% average accuracy on VisDoMBench with Qwen3-VL-32B, and 40% to 74% on SlideVQA ($G^2$-Reader). We will rerun its Lite variant with our backbone and retriever, and bolt the ledger and commit rule onto its worker. Beating it means a gain on the dense-figure subsets (SPIQA, SciGraphQA, FetaTab) under a matched token budget. Ties on text-heavy subsets are acceptable.

On ViDoSeek, the strongest reported number is ViDoRAG with GPT-4o at 73.6% on table-heavy queries versus 61.0% for TextRAG, plus a reported "over 10%" overall margin (ViDoRAG). We rerun ViDoRAG with our open backbone, so the comparison isolates the method rather than GPT-4o. For DocHop-QA, the file gives only generation and localization scores: BLEU 21.56 and ROUGE-L 23.90 for Qwen2.5-VL, and 12.35 F1 for LayoutLMv3 localization (DocHop-QA). We report localization F1 directly from ledger boxes. For DocHop and Reasoning-OCR, no iterative-agent result was found on Lacuna, so our own reruns of IRCoT-style and ViDoRAG-style loops are the strong baselines there.

Two ablations isolate the key ingredient, and both are matched to the full system on vision-language-model calls and image tokens:

  • Always zoom. Re-read every intermediate value at maximum zoom, with no OCR and no check.
  • Self-consistency. Replace GOT with a second sampled reading from the same vision-language model, keeping the commit rule.

A third ablation removes only the label-matching check. This separates value misreads from entity mismatch.

All systems share retriever, backbone, hop cap, and a tuning budget of the same number of development-set configurations.

Why not just…?

Why not just feed all pages to a long-context frontier model? The file's evidence is consistently against it. DocHop's best reasoning model reaches 60.18% against 92.60% for humans on single pages. DocHop-QA found that images hurt multi-document answering. Vanilla GPT-4o gets 8.76% on MMhops Comparison. We still run this baseline with GPT-4o on a subset so that the claim rests on our own numbers.

Why not just run GOT once over the corpus and use IRCoT on the resulting Markdown? This is a real threat on text-heavy documents, and we expect it to be competitive there. The counter-evidence is ViDoRAG's 61.0% versus 73.6% for TextRAG on tables, and $G^2$-Reader's "shredder" finding. More decisively, static OCR has no second reader, so it cannot detect its own misreads. Our cascade-injection experiment applies equally to it. If static-OCR IRCoT matches us on DocHop's 6-chart pages, our claim is wrong.

Why not just zoom in on every intermediate read? If verification works only because it buys resolution, the always-zoom ablation will match it at equal compute, and we will report that resolution, not cross-reader agreement, is the active ingredient. We expect always-zoom to fix digit misreads but not wrong-series binding, which requires the label check.

Why not just sample the vision-language model twice and check agreement? Samples from one model share its blind spots, so we predict their errors are correlated. The error-independence analysis tests this directly. If same-model agreement predicts errors as well as cross-reader agreement does, the OCR model is unnecessary and we will say so.

Proposed Experiments

The main evaluation runs on four benchmarks:

  • DocHop, stratified by charts per page. The headline test is whether the slope from 2 to 6 charts flattens.
  • ViDoSeek and VisDoMBench, for multi-page retrieval.
  • DocHop-QA chain-hop versus fan-hop questions. We predict gains concentrate on chain-hop.
  • Reasoning-OCR, as a check that verification does not cost reasoning ability, including on its decision-reasoning category where models often score 0%.

The decisive checks are threefold.

  • Error independence. On DocHop regions with ground truth, the rate at which both readers are wrong and agree must be well below either reader's individual error rate. If it is not, the commit rule cannot help, and we stop.
  • Disagreement as a predictor. Disagreement must predict an incorrect final answer, measured by area under the ROC curve on held-out questions.
  • Cascade suppression. Under injected first-hop errors, accuracy with the commit rule should degrade markedly more slowly than without it. It should also beat both matched-compute ablations.

We also report cost per question and re-read frequency, since an approach that re-reads nearly every value is just always-zoom with extra steps.