A PDF parser has to turn a two-dimensional page into one stream of text. On a newspaper, a multi-column paper or a form with floating boxes, it often gets the order wrong. It jumps from the bottom of column one to a sidebar, or splices a caption into the middle of a sentence. Current systems predict the order in one shot and never look back. Some output a sequence, some a graph of "this block is read right after that one," some a tree. When an early decision is wrong, the error spreads. Programmers handle the same kind of problem differently. They run tests, read which test failed and where, patch that spot, and revert the patch if it breaks something that used to work. This note proposes giving reading order the same treatment. A candidate order becomes a small program, and each point where one block hands off to the next becomes a unit test. The failing handoffs are repaired locally, and any repair that breaks a passing handoff is rolled back.
The central bet is that reading-order errors are mostly local and independent. A page has a handful of ambiguous junctions, such as a column break, a sidebar, a figure caption or a hyphenated line end, and each can be checked on its own by asking whether the text on the two sides actually continues. If that holds, fixing junctions one at a time should beat resampling whole pages. The reason is the arithmetic of independent errors: a whole-page sample that gets every junction right is exponentially rare, while local repair costs only linear effort in the number of junctions. The proposal is built so that this bet can fail visibly against a compute-matched best-of-N reranker.
Core Claim
We claim that a cheap, localizing verifier plus rollback-guarded local edits improves reading order on dense prose layouts beyond what the underlying parser achieves. We also claim it beats best-of-N reranking with the same verifier at equal verifier cost, and that the advantage grows with the number of ambiguous junctions per page.
The counting argument behind the scaling prediction is simple. Suppose a page has $k$ ambiguous junctions and a sampler resolves each one correctly with independent probability $p$. Then a full-page sample is entirely correct with probability $p^k$, so best-of-N needs on the order of $p^{-k}$ samples. A repair loop that tests and fixes each junction separately needs on the order of $k/p$ attempts. The independence assumption is the thing to test. If junction errors are strongly correlated, for example when a parser misreads the whole column structure at once, then the two costs converge and the claim weakens.
The closest existing work is Vision-Guided Iterative Refinement for Frontend Code Generation. It renders generated HTML and sends a vision-language model's critique of the screenshot back into code revisions. That paper judges whole pages for aesthetic quality, its critic gives no location for the error, and its ablation reports only a 1.5% gain when refining without the critic. Our loop differs in three ways. It critiques structure rather than appearance. Its feedback points to a specific junction rather than to the whole page. Its "tests" are computed from the document's own text rather than produced by a judge model.
The other near neighbor is Reading or Reasoning? Format Decoupled Reinforcement Learning for Document OCR, which uses feedback only offline, as a training reward, and then decodes in a single pass. We work at inference time, on top of any parser, including an RL-trained one.
Research Program
The program representation. A candidate reading order is written as a short sequence of statements over detected block identifiers. Each statement either emits a block into the main flow, attaches a block as an aside to a parent, or marks a block as a float, such as a caption or footnote, anchored to a position. This borrows the transition vocabulary of PDF-to-Tree: Parsing PDF Text Blocks into a Tree, which parses blocks into a document tree with shift-and-attach actions. PDF-to-Tree commits greedily; here its actions become editable code. The representation also covers the non-linear, graph-shaped orders of Modeling Layout Reading Order as Ordering Relations for Visually-rich Document Understanding. That work showed that 23.76% of segments in its ROOR annotations are involved in non-linear order, so a flat permutation is not enough.
The tests. "Compiling" the program checks hard constraints: every block is covered exactly once, the flow has no cycles, and every heading precedes its children. "Running" it scores each handoff from block $i$ to block $j$ with a continuation test:
$$s(i,j) = \log p_{\mathrm{LM}}(\mathrm{head}_j \mid \mathrm{tail}_i) - \log p_{\mathrm{LM}}(\mathrm{head}_j \mid \varnothing).$$
Here $\mathrm{tail}_i$ is the last few dozen tokens of block $i$, $\mathrm{head}_j$ is the first few dozen tokens of block $j$, $p_{\mathrm{LM}}$ is a small frozen language model, and $\varnothing$ is an empty context. The subtraction matters. The test asks whether block $j$ becomes more predictable after block $i$, not merely whether it is fluent text. This turns the linguistic-continuity idea of Combining Linguistic and Spatial Information for Document Analysis into a per-handoff score. It also addresses the failure that the RORE paper names in its own model: relation predictors lean too heavily on spatial proximity.
A handoff fails its test when some geometrically plausible alternative successor $j'$ scores clearly higher, meaning $s(i,j') - s(i,j)$ exceeds a margin tuned on a development split. Small signals get dedicated tests. A line ending in a hyphen or with no sentence-final punctuation must continue to a block that begins in lowercase or mid-clause, unless the next block is a float. Each test failure works like a traceback: it names the two blocks and the region of the page.
The repair loop. For each failing handoff, a repairer proposes an edit restricted to a small window of neighboring blocks. The edit can swap two blocks, move a block, or reclassify a block as an aside or float. We will compare two repairers: plain enumeration over the window, and an LLM that is shown the failing test, the window's text and a crop of the page image. An edit is accepted only if the failing test now passes and no previously passing test fails. Otherwise it is rolled back. This is the regression-guard design of MEIC: Re-thinking RTL Debug Automation using LLMs, whose rollback to the last stable version stops the "new error traps" that unconstrained LLM fixers fall into.
Because acceptance is monotone in the test suite, the loop cannot drift. This speaks directly to the plateau-and-degrade pattern that Yan et al. report for iterative refinement. The loop can still stall, and whether it stalls at an accurate order is an empirical question.
The same mechanism supplies the backtracking that LayerDoc: Layer-wise Extraction of Spatial Hierarchical Structure in Visually-Rich Documents names as missing. LayerDoc groups words into lines, lines into regions and regions into blocks, and its authors observe that an early grouping error permanently distorts the levels above it. A failed handoff test at block level is exactly the higher-level evidence that should trigger revision of a lower-level grouping. We will add one edit type that splits a block when a test fails in the middle of it.
Scope and an honest limit. The continuation test has force only where text actually runs across blocks: prose in newspapers, magazines, academic papers and textbooks. On forms, many key-value handoffs carry little linguistic signal. We therefore treat ROOR and EC-FUNSD as a stress test of where the method stops helping, not as its home turf. The expectation is that gains there will be small unless geometric and field-type tests are added.
Baselines
The naive reference point is the base parser's own output with no further processing, plus a geometric order from Augmented XY Cut, the recursive coordinate-splitting approach used in XYLayoutLM: Towards Layout-Aware Multimodal Networks For Visually-Rich Document Understanding. The repair loop runs on top of each and must improve both. XY Cut is the cheapest serious competitor on multi-column prose. If it is already near ceiling there, the problem is smaller than claimed.
The strongest existing parsers on OmniDocBench are the reference systems. The best overall number in the file is FD-RL's average score of 90.41 on OmniDocBench, reported in Reading or Reasoning? Format Decoupled Reinforcement Learning for Document OCR. The same paper reports 88.41 for dots.ocr and 87.01 for Deepseek-OCR. Those are aggregate parsing scores. We found no reported result on Lacuna for the reading-order component of OmniDocBench alone, so we will measure it ourselves for every system we can run. We will run the loop on the outputs of the strongest openly available parser among these, using the same page images and the same block detections. Beating it means a statistically significant improvement in the benchmark's reading-order score. The improvement should concentrate on the newspaper and magazine domains that OmniDocBench: Benchmarking Diverse PDF Document Parsing with Comprehensive Annotations identifies as the universal failure mode, and the loop should not lose ground on simple domains.
For hierarchy, the reference is PDF-to-Tree's 93.93% unlabeled attachment score on its own dataset, from PDF-to-Tree. Unlabeled attachment score is the fraction of blocks attached to the correct parent. We rerun that parser and then apply the loop to its trees. For relational order, no reported ISDR-level score on ROOR was found on Lacuna. ISDR is RORE's relation meaning "block j is read immediately after block i." We will train a RORE-style relation predictor on the ROOR split and report edge F1 before and after repair.
The competitor that most directly threatens the proposal is best-of-N reranking with the same verifier. We draw N whole-page orders from the base parser by temperature sampling, or from an LLM given the blocks when the parser is deterministic, and keep the one with the highest total test score. N is chosen so that the number of language-model scoring calls matches the repair loop's on each page. Tuning budgets are identical, and both methods share the margin threshold. Beating it means higher reading-order accuracy at equal calls, with the gap widening as junction count grows.
We also compare against a whole-page visual critic modeled on Vision-Guided Iterative Refinement for Frontend Code Generation. The current order is rendered as numbered overlays on the page image, and a vision-language model is asked to critique and revise it for three cycles.
The ablations isolate the key ingredients. We replace the targeted repair with edits at random locations, keeping the same number of edits. We remove rollback. We replace the mutual-information test with raw likelihood of the joined text. We swap the LLM repairer for plain enumeration.
Why not just…?
Why not just rerank whole-page samples with the same scorer? This is the real threat. If junction errors are strongly correlated, or if pages have only one or two ambiguous junctions, best-of-N with the verifier will match local repair, and the contribution reduces to the verifier alone. That would still be useful, but it is a smaller claim. The compute-matched comparison, stratified by junction count, settles it. The counting argument predicts a widening gap with $k$. A flat gap would falsify the local-repair claim, and we would report the verifier as the finding.
Why not just train a better single-pass parser with RL, as FD-RL does? FD-RL's 90.41 shows that offline reward shaping works for overall parsing. It still decodes once, and its own entropy analysis shows that uncertainty concentrates in structured regions, where a single decode has no chance to recover. The loop is complementary rather than a replacement: we run it on FD-RL-class outputs. If it adds nothing there, the RL-trained parser has already absorbed the benefit. The per-junction test score could also serve as an RL reward, which we treat as follow-up work rather than a competitor.
Why not just ask a vision-language model to critique the rendered order, as in the frontend work? OmniDocBench reports that general models such as GPT-4o and Qwen2-VL-72B fail on dense multi-column layouts, which are exactly the pages where the critic is needed. The frontend paper's gains also depend on a critic signal that is available and informative, as its critic-free ablation shows. The whole-page visual-critic baseline measures this directly. We expect it to help on simple pages and to fail to localize errors on newspapers.
Why not just predict the reading-order graph better in one pass? RORE injects predicted order relations into attention and gains 2.86 F1 on EC-FUNSD entity linking and 6.48 on InfoVQA. It is still open-loop, and its authors name over-reliance on geometry as a remaining failure. The repair loop takes a RORE-style prediction as its starting point. Measuring how many of that predictor's edge errors the loop fixes tells us whether semantic handoff tests supply information the single-pass predictor lacked.
Proposed Experiments
The first decisive check is controlled. We take ground-truth orders from OmniDocBench prose domains and the PDF-to-Tree dataset, inject $k$ independent junction errors for $k$ from 1 to 8, and plot recovery rate against verifier calls for the repair loop and for best-of-N. The prediction is roughly linear cost growth for repair and fast-growing cost for reranking. We then repeat the experiment with correlated errors, such as swapping entire columns, to find where the advantage disappears.
The second check is a verifier audit, run before any search. On gold orders, what fraction of true handoffs pass the continuation test, and what fraction of plausible wrong handoffs fail it, broken down by domain? If the gold-pass rate on newspapers is low, the method cannot work there, and it is better to learn that early.
The third check is end-to-end. We apply the loop to the naive and strongest parsers on OmniDocBench and report the reading-order score per domain. We also report PDF-to-Tree attachment score and ISDR edge F1 on ROOR, along with the number of language-model calls and wall-clock time per page, since cost is the standing criticism of refinement loops.
The fourth check is downstream. We feed repaired serializations into a fixed question-answering reader on InfoVQA, which the RORE line shows is sensitive to order. This tests whether better handoffs translate into better answers.
References
- Vision-Guided Iterative Refinement for Frontend Code Generation
- Reading or Reasoning? Format Decoupled Reinforcement Learning for Document OCR
- PDF-to-Tree: Parsing PDF Text Blocks into a Tree
- Modeling Layout Reading Order as Ordering Relations for Visually-rich Document Understanding
- Combining linguistic and spatial information for document analysis
- MEIC: Re-thinking RTL Debug Automation using LLMs
- CodeTransOcean: A Comprehensive Multilingual Benchmark for Code Translation
- LayerDoc: Layer-wise Extraction of Spatial Hierarchical Structure in Visually-Rich Documents
- XYLayoutLM: Towards Layout-Aware Multimodal Networks For Visually-Rich Document Understanding
- OmniDocBench: Benchmarking Diverse PDF Document Parsing with Comprehensive Annotations