Born-Digital PDFs as Free Verifiers for Multimodal Reasoning

Reinforcement learning on problems with automatically checkable answers has been the main driver of recent reasoning gains in language models. A model trained this way only on math gets better at code and science too. Multimodal models have struggled to follow, because checkable visual problems are scarce: someone has to write the question, the answer, and ideally the image region that justifies it. We propose a cheap source of such problems. Born-digital PDFs, meaning PDFs generated by software rather than scanned, carry a hidden layer that records every character, number, and form field together with its exact page coordinates. Render the page to pixels and show the model only the image. The hidden layer can then grade a multi-step question about that page: it checks the final answer, every intermediate value, and every region the model claims to be looking at.

The core claim is that this hidden layer turns web-scale PDF crawls into a source of dense, step-level rewards for multimodal reinforcement learning. Training on these document-reasoning tasks should improve visual math and science benchmarks the model never trained on, and should do so more than the same compute spent on curated grounding data or on synthetic pages. A secondary claim is mechanistic. The step-level grounding reward gives a direct training signal to a small gate that decides how much the model relies on a few image-wide "summary" tokens versus local patch tokens. That gate should be what keeps visual attention from fading over long reasoning chains.

What exists and what is missing

Distant supervision means generating labels automatically by aligning raw data with existing structure instead of asking humans to annotate. CommonForms applies it to documents: it filters 7.9 million Common Crawl PDFs for embedded interactive form objects and gets 480,000 pages of form-field boxes with no human labeling. FFDNet-Large trained on these boxes reaches 81.0% mAP50-95. The supervision is single-step, though. It says where a box is and says nothing about reasoning over the page, and the paper itself leaves the relating of fields to their labels as future work. We reuse its harvesting idea and its 14-domain split, and we extend the supervision from flat boxes to verifiable multi-step chains. VSR shows that weighting visual against textual features pixel by pixel helps layout analysis. It needs extracted text as an input and is trained only with detection losses. We use the text layer only as a grader and never as an input.

On the reinforcement-learning side, transfer across domains is well documented in text. CPL trains on math plans and gains 12.2% on HumanEval and 8.6% on GPQA. One-shot RLVR improves unrelated domains from a single geometry problem. Satori shows that self-correction behaviors learned this way transfer to logic and code. These papers motivate the transfer hypothesis but are text-only. Two multimodal papers are the closest prior work and define our boundary. SATORI-R1 has the model emit a caption, then a bounding box, then its reasoning, and rewards box overlap and answer accuracy. Its 12,000-sample VQA-Verify set is reformatted from 17 existing annotated benchmarks, and its only intermediate step is one box. StructVRM scores sub-questions separately but uses a learned neural verifier and curated chain-of-thought traces. Our proposal differs from both in three ways. Supervision is mined from raw, unannotated documents at a scale two orders of magnitude larger. Every intermediate step, not only one box, has a deterministic check. And the grader is exact geometry and arithmetic rather than a model.

The mechanistic piece builds on To Sink or Not to Sink. That paper found that a few high-norm "sink" tokens in Vision Transformers (image encoders that split an image into patches and let every patch attend to every other) hold compressed summaries of the whole image. Pushing those tokens and ordinary patch tokens through one shared projector causes interference. Its DIYSink fix uses separate projectors plus a query-conditioned gate and gains up to 7.5 points on MathVista geometry, but it is trained only with supervised fine-tuning. Grounding-focused RL such as SATORI-R1 treats all patches the same. No one has trained the sink/patch gate with a reward that directly measures whether the model looked at the right local region. That is what our step-level box reward provides.

Research Program

The first component is a data engine. We crawl born-digital PDFs and keep pages whose text layer is trustworthy. The test is to render the text layer alone, render the full page, and require that each extracted token is visible and non-occluded in the full render. This rejects the hidden text, decorative form boxes, and drift problems that CommonForms reports. From the surviving pages we recover tables by clustering token coordinates into rows and columns, and we recover form structure by linking AcroForm fields to their nearest label text.

Templates then compose multi-step questions over these structures. For example: find the row labeled X, read the value in column Y, subtract the value two rows below, and report whether the result exceeds the total in the footer. Other templates cover cross-references between a form label and its field, and comparisons that span reading order. Each question stores its chain of intermediate values, the source box for each value, and a final answer. We target roughly one to two million questions spanning the 14 CommonForms domains, including the one-third of pages that are non-English. Scanned PDFs that carry an OCR text layer go into a separate, explicitly noisier tier, so we can measure how label noise changes the result.

The second component is training. A 7B open vision-language model is trained with Group Relative Policy Optimization (a policy-gradient method that scores each sampled answer against the other samples for the same prompt). The model outputs a sequence of steps, each consisting of a box and a value, followed by an answer. Step $k$ has a predicted value $\hat v_k$ and a predicted box $\hat b_k$, which are compared against the mined value $v_k$ and box $b_k$. The reward is

$$ r = \mathbb{1}[\hat a = a] + \frac{\lambda}{K}\sum_{k=1}^{K}\Big(\mathbb{1}[\hat v_k = v_k] + \mathrm{IoU}(\hat b_k, b_k)\Big), $$

where $a$ is the true final answer, $\hat a$ the predicted one, $K$ the number of steps, and $\lambda$ the weight on intermediate credit. Every term is computed exactly from the PDF layer. Following DIYSink, the visual connector has separate projectors for sink and patch tokens and a small gate that mixes them conditioned on the query. Here, though, the gate is trained end to end by $r$.

The third component is analysis. We track how much attention the model pays to patch tokens as reasoning chains get longer, which is the drift SATORI-R1 calls visual-attention deficiency. We also track the gate's weights across question types, and we test whether gains on held-out benchmarks track the strength of in-domain grounding.

Baselines

The do-nothing reference is the untrained base model, evaluated with the same prompting and decoding settings as every trained variant.

The strongest existing method is SATORI-R1. It is the closest competitor because it already combines verifiable RL with explicit grounding and reports SATORI-7B at 76.2% on MathVista. We will cite that number, but its base model and training budget may differ from ours. For the main comparison we will therefore retrain the SATORI recipe on VQA-Verify from our own base model, with the same rollout count, GPU-hours, and hyperparameter search budget as our method. Beating it means a higher MathVista score than the retrained SATORI model at equal compute, with the gap larger than seed-to-seed variance across three seeds.

For ScienceQA, the reported reference point is Seed-StructVRM at 95.1% accuracy. StructVRM's base model is much larger, so we treat this as a ceiling to report rather than a matched comparison. There is no reported result on Lacuna for Attention-Guided Switching on MathVista or ScienceQA. For that method we will use its reported +15.9-point MathVerse gain on the 4B variant as context and rerun it training-free on our base model.

Four further baselines separate the contribution of each ingredient. A text-only RLVR baseline trains the same model on math text with matched compute, in the style of CPL. It tests whether visual training is needed at all for visual-math transfer. A synthetic-page baseline renders the same number of procedurally generated tables and forms with the same templates and rewards. It tests whether real-document diversity matters. A supervised baseline fine-tunes on the mined chains directly instead of using RL. Finally, ablations remove the intermediate terms (set $\lambda=0$) and replace the dual projector and gate with a single shared projector. These isolate the step-level reward and the reward-trained gate.

Why not just…?

Why not just train on SATORI-R1's VQA-Verify data? It is clean, but it is small and borrowed. Its 12,000 samples come from 17 benchmarks, several of them document benchmarks, so its supply cannot grow, and its only intermediate check is one box per question. If the retrained SATORI baseline matches our method at equal compute, then scale and multi-step supervision do not matter for transfer, and we will report that. Data mined from PDFs would still have the advantage of being unlimited and unlabeled. We will also train on a VQA-Verify-sized subset of our data to separate the effect of data source from the effect of data size.

Why not just render synthetic tables and forms? This is the most serious threat. Procedural generation gives perfect labels, with none of the hidden-text or decorative-field noise that CommonForms documents. The case for real PDFs is diversity: layouts in 14 domains, multilingual text, and real typography. Whether that diversity matters is an open empirical question. The matched synthetic-page baseline settles it. If synthetic pages transfer as well as real ones, the finding becomes "step-level verifiability, not realism, drives transfer." We would then pivot the data engine to cheap synthesis rather than defend the mining pipeline.

Why not just do text-only math RLVR and rely on transfer? CPL and one-shot RLVR show strong transfer within text, and a multimodal model's language backbone might carry those gains into visual math. However, SATORI-R1 reports that long text-only reasoning makes attention drift away from task-relevant patches, and that grounding the reasoning gives up to 15.7% on MMBench over free-form R1-style training. Text-only RL plausibly improves the logic but not the reading of the image. Our matched text-only baseline, combined with the attention-drift measurement, tests this directly. The two should diverge most on MathVista chart and figure items and on POPE, a benchmark that measures object hallucination.

Why not just use answer-only rewards, since one-shot RLVR works with them? One-shot RLVR shows that outcome rewards suffice for text math. In multimodal RL, however, outcome-only reward cannot tell a correct answer from a correctly read page. The $\lambda=0$ ablation measures exactly how much the dense terms add. If they add nothing, the method reduces to a large answer-checked document dataset, which is still useful but makes a smaller claim.

Why not just keep reasoning in latent space, as Attention-Guided Switching does? That method reports large efficiency gains from skipping explicit text during perception-heavy steps. Latent steps, however, cannot be checked against coordinates. We choose explicit boxes so that every step stays gradable. We will report generation length next to accuracy so the cost of this choice is visible.

Proposed Experiments

The decisive test has three parts, all at equal compute. First, train our model, retrained SATORI, the synthetic-page baseline, and the text-only baseline. Second, evaluate all of them on benchmarks with no document content in training: MathVista (including its geometry and function subsets), MathVerse, MathVision, and ScienceQA. Third, evaluate on POPE for hallucination. In-domain generalization is measured on held-out CommonForms domains and held-out non-English pages. We also check, with string matching, for overlap between our mined pages and DocVQA-style sources.

The mechanistic check has two predictions. With the gate trained by the grounding reward, attention to patch tokens should decay more slowly over long chains than with the shared projector. The gate should also shift toward patch tokens on counting and lookup steps. If neither prediction holds, we drop the gate and keep only the data contribution. A final noise study compares the clean born-digital tier against the tier with OCR text layers, which shows how much label noise reinforcement learning tolerates before transfer disappears.