When a language model answers a question about a 100-page annual report, it usually fails in one of two ways. It never finds the right number, or it finds a number that looks right but comes from the wrong table, the wrong year, or the wrong business segment. Letting the model "think step by step" (chain-of-thought prompting, where the model writes intermediate reasoning before its answer) does not reliably fix either failure. Free-form reasoning over noisy text can talk the model into false connections. We propose a stricter kind of reasoning trace. Every step must first name where in the document its evidence lives, as a path through the document's own structure, for example Item 7 > Liquidity > Table 3 > row "Net cash from operations" > column "2019". Only then may the step use the value found there. A deterministic checker confirms that the address exists and that the quoted content matches it. The model may also declare that no address supports the question and decline to answer.
Addresses of this kind make two things measurable that are currently tangled together. The first is whether the model can navigate to the right place. The second is whether it can compute correctly once it is there. Separating them lets us test a specific, unresolved claim about domain-specific pre-training, meaning continued training of a model on text from one field before task training. Such pre-training is widely believed to help, yet on financial numerical reasoning it has shown no advantage (FinQA found FinBERT no better than RoBERTa). Our hypothesis is that domain pre-training helps navigation (knowing that "net change" lives under a particular header and implies subtraction) but not arithmetic. It should help most when that pre-training exposes the model to filings serialized with their structure intact.
Core Claim
Chain-of-thought helps under contextual noise when each step is anchored to a verifiable structural address, and hurts when it is not. Two findings in the literature appear to conflict, and this condition is what separates them. Rethinking Reasoning in Document Ranking shows that free <think> traces degrade reranking: expected calibration error rises from 0.105 to 0.151 and false positives increase in noisy candidate pools. Ext2Gen shows the opposite: a model trained to extract evidence sentences before answering resists up to 25 distractor chunks. We read the difference as a matter of commitment. Ext2Gen's trace commits to specific text, and free thought does not. Sentence extraction still discards the structural context that separates a 2019 cell from a 2020 cell, or a segment table from a consolidated one. The most damaging distractors in filings are exactly these: near-duplicate structural siblings. Addresses carry that context by construction.
The second claim concerns domain pre-training. Continued pre-training on filings serialized as structure-tagged trees should raise address accuracy but leave accuracy given the correct address largely unchanged. That result would reconcile the FinQA parity finding with the broader evidence that domain pre-training matters (PEFT Python study, MonoCoder).
Relation to Prior Work
Structure-aware segmentation supplies our input representation, not our contribution. HiChunk fine-tunes a 4B model to predict section hierarchy and merges retrieved child chunks upward into parent sections. LongRefiner parses documents into XML trees and prunes nodes per query. Both hand the pruned context to an ordinary reader and never ask the reader to reason over the tree. We reuse their trees and make the tree the vocabulary of the reasoning trace.
LiteCoST is the closest method. It trains small models, with supervised fine-tuning and then group relative policy optimization (a reinforcement-learning method that rewards the best of several sampled outputs) using process rewards, to rewrite dispersed text into an intermediate table before answering. It reports large gains on Loong Finance (+27.6 points for LLaMA-3B). It also reports a drop on "spotlight locating," because serializing text into a new table loses local phrasing. Our trace builds no new structure. It points into the document's existing structure and keeps the original content, which is why we expect it to avoid the spotlight penalty. Whether that expectation holds is tested directly.
DocVAL supplies the training recipe. It filters teacher traces with a rule-based validator of bounding boxes before distillation. Our validator checks tree addresses instead of pixels, and it works on multi-page documents where DocVAL covers single pages.
On the noise side, three papers motivate the design. UDA documents GPT-4 collapsing from 45.9 on FinHybrid with retrieval to 0.4 on the full report. DocFinQA shows large drops when FinQA questions are embedded in full 123k-word filings. DAS shows that allowing an explicit "unanswerable" output reduces distraction.
FastDoc is a constraint and a partial method source. It shows that document-level structural supervision can replace much of the cost of token-level continued pre-training in encoders. It leaves decoder-only models to future work, so we test only the simplest decoder analogue: next-token training on structure-tagged serializations.
Research Program
The pipeline has three pieces.
The first is a document tree built from EDGAR HTML filings. The HTML gives headings and table markup directly. We use a HiChunk-style segmenter only for regions where the markup is missing. Table cells become leaves whose address includes the table caption, row header and column header.
The second is a reader, a 3B–7B open decoder model, trained to emit traces of the form "address → quoted content → operation." A trace may instead end in an abstention step when no node supports the question. Training data comes from a strong teacher prompted to produce such traces on FinQA and DocFinQA questions. The checker keeps only traces whose addresses resolve, whose quotes match, and whose program executes to the gold answer. We then run reinforcement learning with a reward that combines answer correctness with the fraction of steps whose addresses validate.
The third piece is the domain pre-training study. Starting from the same base model, we compare three conditions under a matched token budget: no continued pre-training, continued pre-training on plain-text 10-K filings, and continued pre-training on the same filings serialized with their tree tags. All three then receive identical reader training.
The analysis decomposes end-to-end error into two parts. The first is navigation error, where the model cites wrong addresses. The second is computation error, meaning a wrong answer despite gold addresses. We measure the second directly by an oracle-address condition that supplies the gold nodes.
Baselines
The naive reference points are two pipelines. One feeds the full filing to a long-context model with direct answering, the setting in which UDA reports GPT-4 at 0.4 on FinHybrid. The other is flat 512-token chunking with BM25 retrieval and ordinary chain-of-thought. We include BM25 because UDA finds it beats dense retrieval on financial identifiers and dates. Beating these is necessary but uninformative.
The strongest existing method on this question is LiteCoST's structured-thought training (LiteCoST). It was built for exactly this regime: small models reading long financial documents, with process-rewarded intermediate structure. We reimplement it on the same base model, the same retrieved tree nodes, the same number of training questions and the same reinforcement-learning steps, and tune both methods on the same validation split. The file reports LiteCoST only as gains on Loong Finance (+27.6 for LLaMA-3B, +17.8 for Qwen-7B); no reported absolute result was found on Lacuna for LiteCoST on FinQA or DocFinQA, so the comparison must be run rather than quoted. Beating LiteCoST means matching or exceeding it on overall execution accuracy on DocFinQA, while clearly exceeding it on the structural-distractor and single-sentence "spotlight" subsets. Those subsets are where the two approaches make different predictions.
The second strong competitor is Ext2Gen-style extract-then-answer (Ext2Gen). It reports an average of 0.557 against 0.473 for default Llama3.1-8B on its own open-domain noise benchmark, a number that does not transfer to our setting. We apply it to the same hierarchical retrieval output. On FinQA's snippet setting, the published reference is FinQANet at 61.24% execution accuracy, with finance experts at 91.16% (FinQA). For retrieval-side comparison we include HiChunk auto-merge retrieval feeding a free chain-of-thought reader.
Three ablations isolate the key ingredient. The first removes addresses but keeps the quoted content, which is exactly sentence and cell extraction. The second keeps addresses but disables the checker in both filtering and reward. The third replaces the address-grounded trace with free chain-of-thought under identical training.
Why not just…?
Why not just prepend each chunk's section and table path and let ordinary chain-of-thought run? This is the most serious threat. It is cheap, needs no special training, and gives the model the same structural context. If it matches our method, the contribution shrinks to "structure is useful metadata." We treat it as a first-class baseline, with header-path prefixes on every retrieved node, trained with the same data and reward minus the address-validity term. Our prediction is that it will tie on clean questions and lose on adjacent-period and sibling-segment distractors. The model can read the prefix, but nothing forces it to commit to one. The calibration failure reported in Rethinking Reasoning in Document Ranking arises in just this uncommitted mode. If the prediction fails, we will report that the checker adds nothing beyond metadata.
Why not just use Ext2Gen-style evidence extraction on hierarchical chunks? Extraction already gives noise robustness on flat open-domain text. However, a table cell extracted without its row and column headers is ambiguous by construction, and structural siblings are the dominant distractor in filings. The addresses-removed ablation is precisely this alternative, so the experiment settles it directly. If the gap between the two is small on the sibling-distractor subset, addresses do not earn their cost.
Why not just convert the retrieved material into a table first, as LiteCoST does? Building a new structure throws away phrasing, and LiteCoST itself reports degraded spotlight locating. Pointing into existing structure should not pay that price. The spotlight subset and the head-to-head at matched budget test this. If LiteCoST wins overall and ties on the spotlight subset, our method's advantage is confined to calibration and abstention, and we would say so.
Why not just do domain continued pre-training, or use a larger long-context model? Long context collapses on this task (UDA; DocFinQA), and FinQA found domain pre-training alone gave no reasoning gain. The three-arm pre-training study measures what domain pre-training does and does not contribute once navigation and computation are separated.
Proposed Experiments
The main benchmark is DocFinQA, whose full filings supply natural distractors, with FinQA used for snippet-level comparison to published numbers and UDA's FinHybrid and TatHybrid used for transfer to PDF-derived structure. HiCBench and Gov-Report check that the tree builder is good enough; HiChunk reports 0.9882 segmentation F1 there, and our parser should be close before downstream results are trusted.
We construct three diagnostic subsets from DocFinQA. The structural-distractor subset places the gold table next to same-caption tables from other years or segments. The spotlight subset contains questions whose evidence is a single sentence. The unanswerable subset is built by deleting the gold node from the tree.
The metrics are:
- execution accuracy;
- address precision and recall against FinQA's gold supporting facts, mapped into the tree;
- accuracy given oracle addresses;
- abstention quality on the unanswerable subset, measured by area under the ROC curve (how well the model's willingness to answer separates answerable from unanswerable questions) and by expected calibration error;
- reasoning tokens per question.
Three results are decisive, and each can falsify the proposal.
First, on the structural-distractor subset, address-grounded traces must beat both free chain-of-thought and the header-prefix baseline. On the unanswerable subset, they must reduce false answers relative to free chain-of-thought, whose calibration we expect to degrade as in the reranking study. If the header-prefix baseline ties on both, the address and checker mechanism is unnecessary.
Second, against LiteCoST at matched budget, our method must not lose on the spotlight subset. If it does, pointing into the source offers no advantage over rebuilding it.
Third, in the pre-training study, structure-tagged continued pre-training should raise address recall more than plain continued pre-training does, and neither should noticeably change oracle-address accuracy. If domain pre-training instead improves oracle-address accuracy, the navigation-versus-computation account of the FinQA parity result is wrong. That would be an informative result in its own right.
References
- FinQA: A Dataset of Numerical Reasoning over Financial Data
- Rethinking Reasoning in Document Ranking: Why Chain-of-Thought Falls Short
- Aligning Extraction and Generation for Robust Retrieval-Augmented Generation
- A Comparative Study of PEFT Methods for Python Code Generation
- MonoCoder: Domain-Specific Code Language Model for HPC Codes and Tasks
- HiChunk: Evaluating and Enhancing Retrieval Augmented Generation with Hierarchical Chunking
- Hierarchical Document Refinement for Long-context Retrieval-augmented Generation
- Long-Document QA with Chain-of-Structured-Thought and Fine-Tuned SLMs
- DocVAL: Validated Chain-of-Thought Distillation for Grounded Document VQA
- UDA: A Benchmark Suite for Retrieval Augmented Generation in Real-World Document Analysis
- DocFinQA: A Long-Context Financial Reasoning Dataset
- Improving Zero-shot Reader by Reducing Distractions from Irrelevant Documents in Open-Domain Question Answering
- ***FastDoc***: Domain-Specific Fast Continual Pre-training Technique using Document-Level Metadata and Taxonomy