When a language model reasons step by step toward an answer, the intermediate steps make different kinds of claims. Some are facts ("the bridge is 1,280 meters long"). Some are computations ("so the round trip is 2,560 meters"). Some are logical moves ("therefore the second route is shorter"). Each kind fails in its own way. Facts made up from memory are best caught by looking them up in documents. Computations done "in the head" are best caught by running code. Current systems usually attach one fix to the whole chain: retrieve documents once at the start, or write the whole solution as a program. This proposal asks whether hallucinations in reasoning chains can be reduced by labeling each step with the type of claim it makes, checking it with the matching tool (a retriever for facts, an interpreter for arithmetic), and letting the model continue only after the check passes.
The motivation is error propagation. In chain-of-thought prompting, where the model writes out intermediate reasoning before answering, each step becomes context for the next. A false fact in step one gets treated as settled and then rationalized in later steps. Checking the whole chain after it is finished comes too late. Retrieving once before reasoning cannot anticipate facts the chain only turns out to need in step four. Our claim is that most remaining errors on problems that mix lookup and calculation come from two sources: claims checked by the wrong tool, and claims never checked before later steps depend on them. Both can be measured directly.
Core Claim
On problems that need both facts from documents and exact calculation, the key design choice is not whether the model has tools. It is whether each step is checked by the tool suited to its claim type before that step is committed. We predict three effects compared with the same tools used without typing. Typed checking before commitment should lower the rate at which errors first enter the chain. It should also cut the share of wrong answers traced to an early wrong step. It should do both at lower tool cost than checking every step with every tool. We also make a narrower, falsifiable prediction: when the tools are identical, a free agent that decides for itself when to call a tool will under-check factual claims. It will call the interpreter when a number looks hard, but it will rarely retrieve for a fact it feels confident about. Those confident, unchecked facts are where hallucinations survive.
What Exists and What This Adds
The research file supports each ingredient separately. For computation, Program of Thoughts Prompting has the model write Python and hands the arithmetic to an interpreter. LILA extends this to symbolic libraries such as SymPy. When Do Symbolic Solvers Enhance Reasoning in Large Language Models? finds that language models handle loose parsing of wording well, while solvers handle hard constraints and search. We reuse this division of labor, but apply it per step rather than per problem. BeyondBench shows that removing execution environments sharply lowers performance on algorithmic tasks, which motivates treating computation as something to check rather than trust.
For facts, the retrieval-augmented generation line in the file conditions answers on retrieved passages to reduce made-up content. Tiberiu Boros et al. have the model generate keywords for full-text search and then re-rank the results. Yun Xiong et al. show that structured grounding and re-ranking make small models more factually robust in long contexts. Both retrieve for a query, not for a claim that appears in the middle of a reasoning chain.
For reasoning structure, several papers find that forcing the model to name the kind of step it is taking reduces downstream errors. Studying the capabilities of Large Language Models in solving Combinatorics Problems posed in Hindi has the model classify the problem's combinatorial subcategory before writing equations. SoS1 requires explicit justification of each step. We take this "name the step type first" idea and use the type to choose a verifier, not just to guide wording.
Agentic systems such as AlphaApollo and the tool-using poker agents in How Far Are LLMs from Professional Poker Players? already give models several tools inside a reasoning loop. The obvious idea at this intersection, which is reasoning interleaved with retrieval and code calls, is therefore largely taken. The section of the research file cataloguing existing combined work was truncated, so we could not verify how thoroughly it covers mixed fact-and-computation settings. We treat free-choice agents as the main competitor rather than claim the space is empty. Our contribution differs from such agents in three concrete ways. We type each claim explicitly. We make checking a condition for continuing rather than an option. And we measure where in the chain errors first enter, broken down by claim type.
Research Program
The system has three parts. First, the model writes one reasoning step at a time. The same model is then prompted to split that step into atomic claims and label each claim as a fact, a computation, or an inference.
Second, each label triggers a specific check. A fact must be supported by a retrieved passage. The model issues a search query and must quote a span, and the check verifies that the quoted span exists in the retrieved text and contains the stated value. A computation must be rewritten as a short code snippet and executed, and its output must match the value in the step. An inference is checked more lightly, by asking whether it follows from already committed claims. We treat this as the weakest link and report it separately.
Third, a step that fails any check is rewritten using the tool output, and only verified steps enter the context for later steps. This rule is what should stop cascades.
To attribute errors cleanly, we build a controlled benchmark with known ground truth for every step. We generate a synthetic corpus of fictional entities with numerical attributes, such as towns with populations, companies with founding years, and products with prices. Because the entities are fictional, the model cannot know these facts from pretraining. Any fact it states without retrieval is a detectable hallucination. Questions require two to four lookups followed by arithmetic or small combinatorial computations. Each question comes with a gold decomposition into typed claims, so we can find the first wrong step in any chain and record its type.
We also build a natural variant. We take GSM8K and LILA problems and move the key quantities out of the question into short documents, turning computation-only problems into lookup-then-compute problems. Contamination is a concern here, because models may have memorized the original problems. We handle it by changing the numbers in the documents, so memorized answers become wrong answers. Our purpose is related to BeyondBench's contamination-resistant evaluation, though our method of perturbing values is our own.
Baselines
The naive reference point is plain chain-of-thought with no retrieval and no tools. On the synthetic corpus it should fail almost completely, which confirms that the facts cannot be recalled. On the unmodified computation benchmarks it anchors how much tools add. The strongest reported chain-of-thought number in the file is from Achieving >97% on GSM8K, which reports above 97% accuracy on GSM8K through deeper problem understanding. We will cite that figure for original GSM8K, but it is not comparable on our document-dependent variant. We found no reported result on Lacuna for any method on lookup-then-compute variants of GSM8K or LILA, or on synthetic fictional-entity reasoning.
The second baseline is retrieve-once, then reason. The question is used to retrieve passages up front, using the keyword-plus-re-ranking pipeline of Tiberiu Boros et al., and the model then reasons with chain-of-thought over them. This is the standard way retrieval-augmented generation meets chain-of-thought. It should fail when later hops depend on earlier answers.
The third baseline is program-only reasoning in the style of Program of Thoughts, with retrieval exposed as a callable function inside the program. This is the strongest single-tool design.
The fourth and strongest competitor is a free-choice agent. It has exactly the same retriever, interpreter, and model as our method. It interleaves reasoning with tool calls whenever it chooses, following the agentic pattern of the systems cited above. It is the strongest competitor because it has every capability ours has and differs only in the typing and the requirement to check before continuing.
Two ablations isolate our ingredients. The first keeps typing but checks every step after the chain is complete, which removes the commit rule. The second requires a check at every step but sends every claim to both tools, which removes typing and uses the maximum tool budget.
For fairness, all systems use the same base models: one open model of moderate size and one larger model, both run with the same decoding settings. All share the same retriever and index and the same interpreter sandbox. All receive the same number of prompt-tuning iterations on a held-out development split. We report accuracy against total tool calls and total generated tokens, so that no method wins simply by spending more.
Beating the free-choice agent means two things. On the synthetic benchmark, it means a higher final accuracy and a lower rate of chains whose first error is an unchecked fact, at equal or lower tool calls. On the natural variants, the accuracy gain must hold across both base models, measured with paired bootstrap over questions.
Why not just…?
Why not just write the whole solution as a program with a retrieval function? This is a real threat. Program-of-thought reasoning already sends arithmetic to the interpreter, and a retrieval call inside code could handle facts. The weakness is that the model still decides which values to fetch and which to hard-code. We expect confident parametric facts to be written in as literals rather than looked up. Our synthetic benchmark measures this directly by counting literals in generated programs that match no retrieved value. If program-only reasoning matches our method on both accuracy and first-error rate, we will conclude that typing adds nothing beyond code and say so.
Why not just retrieve once at the start and reason over what comes back? Multi-hop questions often require a second lookup whose target is only known after the first answer, for example the population of the town where the company was founded. Retrieval driven only by the question cannot anticipate this. The file's evidence on positional attention bias, from Yun Xiong et al., further suggests that loading many passages up front makes later evidence harder for the model to use. The comparison on questions with three and four hops will show whether this gap is large or negligible.
Why not just give an agent the tools and let it decide? This is the most serious alternative, and the controlled comparison is built to settle it. If free agents already retrieve for most factual claims, the commit rule and typing will add little. The prediction that would falsify our claim is that the free agent's share of unchecked factual claims is already near zero on fictional entities. We will report that share directly.
Why not just sample many chains and take a majority vote? Voting helps when errors are random. A hallucinated fact about a fictional entity, however, tends to be a stable guess repeated across samples rather than noise that averages out. We include majority voting over the same token budget as an extra comparison, specifically to test whether agreement among samples is correlated with correctness on fact claims.
Proposed Experiments
The first decisive check is error attribution on the synthetic benchmark. For every wrong answer under every system, we locate the first wrong step using the gold typed decomposition and record whether it was a fact, a computation, or an inference, and whether it was checked. Our hypothesis predicts that under the free-choice agent, most first errors are unchecked facts. Under our method, the remaining first errors should shift toward inferences, which is the claim type we check most weakly.
The second check tests the commit rule. We compare typed checking before continuing against typed checking after the chain is complete, using the same tool calls. The fraction of chains where a detected early error still corrupted the final answer measures how much cascading the commit rule prevents.
The third check examines cost. We plot accuracy against tool calls for typed checking, check-everything-with-both-tools, and the free agent. If typing matters, it should reach the accuracy of the check-everything ablation at a fraction of the calls.
The fourth check tests whether the findings transfer. We repeat the comparison on the document-dependent GSM8K and LILA variants with perturbed numbers. We also run the BeyondBench algorithmic tasks, to confirm that the computation checks do not lose accuracy when no retrieval is needed. That last condition guards against the overthinking cost of adding checks to problems that do not need them.
The typing step itself is error-prone, since a mislabeled claim goes to the wrong checker. We therefore report typing accuracy against the gold decompositions and measure how much of the remaining error it explains.
References
- Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Tasks
- LILA: A Unified Benchmark for Mathematical Reasoning
- When Do Symbolic Solvers Enhance Reasoning in Large Language Models?
- BeyondBench: Contamination-Resistant Evaluation of Reasoning in Language Models
- Fine-Tuning and Retrieval Augmented Generation for Question Answering Using Affordable Large Language Models
- U-NIAH: Unified RAG and LLM Evaluation for Long Context Needle-in-a-Haystack
- Studying the capabilities of Large Language Models in solving Combinatorics Problems posed in Hindi
- SoS1: O1 and R1-Like Reasoning LLMs are Sum-of-Square Solvers
- AlphaApollo: A System for Deep Agentic Reasoning
- How Far Are LLMs from Professional Poker Players? Revisiting Game-Theoretic Reasoning with Agentic Tool Use
- Achieving >97% on GSM8K: deeply understanding the problems makes LLMs better solvers for math word problems