Questions about technical documents often need several lookups in sequence. A typical example: a paragraph sets a rule ("only count regions whose 2019 output exceeded the threshold in Figure 2"), the model must read values off one chart, carry them to a second chart, and then compute an answer. Small open vision-language models (models that take images and text and produce text) do very badly here. On DocHop, open models score between 7% and 22%, against 92.60% for humans. The usual way to improve such models is reinforcement learning (RL) on the final answer: sample many attempts and push up the ones that end correctly. This signal is weak exactly where multi-step document reasoning breaks. One misread tick on hop two ruins the chain, and a correct final answer can come from a lucky guess or a shortcut such as the chart title, which models are known to exploit (Plot Twist).
We propose training a compact model on synthetic documents rendered from known underlying data. Because the data behind every chart is known, each intermediate "read" the model makes can be checked exactly, and we reward the model for each one, not only for the final answer. The model must state where it looked (a box on the page) and what value it read. A verifier compares that claim with the data the chart was drawn from. At test time no verifier or chart-to-table tool is needed, because the model reads pixels directly. The resulting claim is narrow and checkable. Dense, exactly verified rewards on intermediate reads should close more of the multi-hop gap than final-answer RL. The advantage should grow with the number of charts and hops, and it should transfer to real documents the model never saw.
Core Claim
Final-answer RL for multimodal multi-hop reasoning cannot tell which hop failed. Training documents built from known data make the intermediate steps of the task automatically verifiable. Rewarding verified reads should do three things. It should improve accuracy most on many-hop, many-chart documents, where top models fall from about 85% on 2-chart documents to about 40% on 6-chart documents (DocHop). It should reduce shortcut answers whose reasoning chain contains wrong reads. And it may make RL viable without a supervised warm-up, a question the literature leaves contested.
Three lines of prior work supply the method. Point-RFT teaches a model to interleave reasoning with (x, y) coordinate tags on a single chart. It then applies Group Relative Policy Optimization, an RL method that compares sampled answers within a group (GRPO), with a format reward and an answer reward. That raises ChartQA from 70.88% after supervised training to 90.04%. Its points are never checked against ground truth, however; they are only a format the model must follow. MMhops-R1 trains a 7B multi-hop agent with web image and text search tools, using a composite reward on answer, tool use and tags. Its hops are coarse search calls over general-domain content, and nothing rewards whether an intermediate finding was right. Think and Answer ME comes closest in spirit. It rewards box overlap and relational consistency for multi-entity grounding in satellite images, but within one image and with no reasoning chain across sources. Our contribution is the reward on intermediate reads. Each read pairs a location with a value and is checked against source data, inside a multi-hop chain that spans several charts and pages.
Other work motivates the proposal by documenting the gap. DocHop-QA finds that giving document images to a model can lower its accuracy compared with extracted text, which the authors call an "alignment tax." It benchmarks the problem but trains nothing. CharXiv and MATH-Vision show that step-by-step reasoning collapses after an early misreading. LLM-Guided Planning over nuclear regulatory documents reaches 81.5% on NuScale-MQA with a prompted GPT-4.1 agent. That agent navigates in text only and calls vision only at the final step. We train compact open weights to use vision at every hop.
Research Program
The first component is a document generator. It samples small tables of data and renders them as multi-page documents with 2 to 8 charts of varied type, rendering library, gridlines and styling. It also writes prose containing rules, cross-references ("see Figure 3") and distractor charts, and composes questions whose solution path is a known sequence of reads. Styling is deliberately varied because Plot Twist shows that irrelevant visual changes such as gridlines degrade accuracy. Because we control the generator, every question comes with three things: the gold evidence path, the exact value at every chart element, and that element's bounding box. To avoid training on the test distribution, the chart types, layouts and task templates are kept disjoint from DocHop's. Some chart types are also held out entirely.
The second component is the output format and the verifier. Inside its reasoning, the model emits read tags, each holding a page index, a box, and a value. A read is counted correct only if two conditions hold. The stated value must match the source datum within a tolerance; we use 5% of axis range, following Chart101's convention in Plot Twist. And the datum's rendered location must fall inside the stated box. Let $c$ be the number of correct reads that lie on the gold evidence path, $|P|$ the length of that path, $w$ the number of incorrect reads, and $K$ the total number of reads. The episode reward is
$$R = \mathbb{1}[\text{answer correct}] + \lambda \frac{c}{|P|} - \mu \frac{w}{K},$$
where $\lambda$ and $\mu$ are fixed weights. The coverage term rewards completing the chain. The penalty term discourages padding the reasoning with confident fabricated reads. This mirrors, at the level of individual steps, the way CARE penalizes overconfident wrong answers. The verifier exists only during training.
The third component is training. The base model is Qwen2.5-VL-7B, the same backbone as MMhops-R1 and Point-RFT. We use a short supervised format stage on generator-produced gold trajectories, followed by GRPO with the reward above. We also run an RL-first variant with no supervised stage, in the style of Metis-RISE. The conflict this variant tests is concrete. Think and Answer ME reports collapse without a cold start, while Metis-RISE reports that a cold start limits exploration. Our hypothesis is that dense read rewards supply the gradient signal whose absence causes the collapse, so RL-first should become viable. If it still collapses, that supports the cold-start camp in a harder setting.
Baselines
The naive reference point is base Qwen2.5-VL-7B, prompted to reason step by step and given whole page images. DocHop places open models at 7–22%. We also report the base model under the exact same input pipeline we use.
The supervised-only baseline trains the same backbone on the same generator trajectories, with read tags included, for the same number of examples. This answers whether the gold paths alone, imitated directly, would suffice. Beating it means a gain on DocHop beyond seed variance, which we estimate from three seeds per condition.
The strongest same-scale competitor is final-answer GRPO with format and answer rewards, as in Point-RFT and MMhops-R1. We run it in two forms, with and without unverified pointing tags. It uses identical documents, an identical supervised stage, identical rollout counts, an identical GPU-hour budget, and the same hyperparameter search. The pointing-but-unverified form is the critical ablation, because it isolates verification from the act of pointing. For reference, Point-RFT reports 90.04% on ChartQA against 83.92% for text-only RL. No reported result was found on Lacuna for final-answer RL on DocHop, so we produce that number ourselves.
The de-rendering baseline converts every chart to a table with the strongest available chart-to-table component. We use a DePlot-style model, the approach behind ChartPaLI in Chart-based Reasoning. A 7B text model is then trained with the same supervised data and the same final-answer RL. We also run an oracle version that receives the generator's true tables at test time, which bounds what de-rendering could achieve.
The frontier reference point is GPT-5.2 Reasoning at 60.18% on DocHop, as reported by DocHop, with Gemini-2.5-Pro-Reasoning at 44.24%. On NuScale-MQA the reference is the prompted planning agent at 81.5% (LLM-Guided Planning). Exceeding 22% on DocHop would beat open models. Approaching 44% would match a frontier reasoning model with a 7B model. We do not claim we will reach 60%.
Ablations isolate each ingredient. One checks values but not boxes. One checks boxes but not values. One drops the coverage term. One adds noisy-verifier control: in a matched run, 50% of read rewards are randomly flipped. If that run gains about as much as the clean run, the benefit comes from extra reward shaping rather than from correctness signal.
Why not just…?
Why not just de-render charts to tables and train a text model? This is the most serious threat. DocHop-QA's alignment tax suggests text can beat images. Two pieces of evidence argue against relying on it alone. Chart-based Reasoning notes that tables lose the color and spatial information many questions depend on. AgenticScholar reports that extracting values from complex plots with optical character recognition tools remains unreliable. Still, synthetic charts are kind to de-renderers, so the comparison that matters is on real figures from CharXiv, DocHop-QA and NuScale. If de-rendering matches us there, we would conclude that the right use of our verifier is to train a better de-renderer, and we would say so.
Why not just prompt a frontier model with a planning agent? On narrow corpora this works well, reaching 81.5% on NuScale-MQA. But that system defers vision to the final step, cannot follow in-text references to tables, and costs $0.215 per query. On DocHop, which requires chart reading at every hop, even GPT-5.2 Reasoning reaches only 60.18%. We report accuracy against cost, and we accept a loss on NuScale if the 7B model wins on reading-heavy questions.
Why not just supervised training on the generated gold paths? The gold paths contain the same information the verifier checks, so imitation might be enough. Prior evidence leans the other way: in Point-RFT and Think and Answer ME, adding RL improved substantially over supervised training alone (70.88% to 90.04%, and 33.34% to 46.72%). In our setting, though, this has to be shown, and the supervised-only baseline with matched data settles it directly.
Why not just add unverified pointing, as Point-RFT does? Pointing alone may account for most of the benefit as a way to anchor the model's attention. The pointing-but-unverified ablation measures exactly that. If it matches the full method, the proposal's central claim fails.
Proposed Experiments
The main evaluation uses DocHop (accuracy, broken down by number of charts and by task category) as an out-of-distribution test. We add DocHop-QA, where the reported Qwen2.5-VL generative baseline is BLEU 21.56 and ROUGE-L 23.90, to test transfer to real PubMed documents, and NuScale-MQA to test transfer to engineering documents. CharXiv reasoning (GPT-4o at 47.1%) and ChartQA serve as single-chart checks that multi-hop training does not hurt basic chart reading. We also report a shortcut rate. On generator-held-out test documents we can compute it exactly: it is the fraction of correct answers whose chain contains at least one incorrect read. On DocHop we approximate it by spot annotation.
Three results decide the question. The first is the slope of accuracy against chart count. Our method should flatten the 2-to-6-chart decline relative to final-answer RL; equal slopes would mean the read reward adds no credit assignment across hops. The second is the gap between read verification and unverified pointing on real-document transfer, and between read verification and the noisy-verifier control. The third is the cold-start comparison: whether RL-first with read rewards trains stably where final-answer RL-first collapses.
The method fails if it improves the synthetic held-out set but not DocHop or DocHop-QA. That pattern would mean the model learned the generator's style rather than how to read charts, and we would report it as a negative result about synthetic verifiable training.
References
- DocHop: Benchmarking Out-of-domain Multi-hop Reasoning in Information-Dense Documents
- Plot Twist: Multimodal Models Don't Comprehend Simple Chart Details
- Point-RFT: Improving Multimodal Reasoning with Visually Grounded Reinforcement Finetuning
- MMhops-R1: Multimodal Multi-hop Reasoning
- Think and Answer ME: Benchmarking and Exploring Multi-Entity Reasoning Grounding in Remote Sensing
- DocHop-QA: Towards Multi-Hop Reasoning over Multimodal Document Collections
- CharXiv: Charting Gaps in Realistic Chart Understanding in Multimodal LLMs
- Measuring Multimodal Mathematical Reasoning with MATH-Vision Dataset
- LLM-Guided Planning for Multi-hop Reasoning over Multimodal Nuclear Regulatory Documents
- CARE: Confidence-Aware Reasoning for Reliable Medical VQA
- Metis-RISE: RL Incentivizes and SFT Enhances Multimodal Reasoning Model Learning
- Chart-based Reasoning: Transferring Capabilities from LLMs to VLMs
- AgenticScholar: Agentic Data Management with Pipeline Orchestration for Scholarly Corpora