When the Answer Key Is Retrieved: Reference-Anchored Step Checking for Retrieval-Augmented Reasoning

An LLM judge grades much more accurately when it is handed a correct reference answer. The reference changes the judge's job from "decide whether this is good" to "spot where this differs from the key." Retrieval-augmented generation (RAG), where a model pulls documents from a corpus and reasons over them, looks like the natural place to cash this in. It rarely has gold solutions, but it always has retrieved text that could act as a reference for each intermediate reasoning step. The catch is that retrieved text is not an answer key. It can be irrelevant, outdated, or wrong. Reasoning models are known to talk themselves into accepting misleading context rather than rejecting it. A judge anchored on a bad reference may simply copy the reference's error, and nobody has measured how often that happens.

This note proposes to measure that failure, which we call anchor capture, and to build a step-level verifier that resists it. The two-sentence version is this. Reference-anchored judging is only safe in RAG if the judge can tell when its reference is lying. We will quantify how badly current judges are captured by corrupted references, then train a verifier that checks each reasoning step against several independently retrieved anchors and can answer "the anchors are unreliable" instead of being forced to agree or disagree.

Core Claim

Our first claim is about diagnosis. The gains reported for reference-anchored judging were measured with trusted references. With untrusted ones, anchoring can make a judge worse than no reference at all. The judge accepts wrong steps that match a corrupted anchor, and rejects correct steps that contradict it.

Our second claim is about method. A verifier trained on deliberately corrupted anchors, with a three-way output (step supported, step contradicted, anchors unreliable), keeps most of the reference-anchoring gain under clean retrieval and degrades gracefully as corruption rises. Unanchored judges and judges that read the generator's own retrieved context do not degrade this way.

The motivation comes from two findings that have not been put together. The first is that references reliably help judges. Withholding references from LLM verifiers costs 5% to 18% accuracy (VerifyBench). Anchoring is argued to work by turning absolute judgment into discrepancy detection (Kim). References strip fluency and length illusions for both LLM and human raters (Doostmohammadi et al., Zhang et al.).

The second is that reasoning models are easily steered by bad context. On SciAux, two misleading snippets drop Qwen3-8B from 96.22% to 77.03% with thinking on, below the thinking-off score of 74.87%. The thinking chain rationalizes the misleading premise (Thinking in a Crowd).

If discrepancy detection is the mechanism, then the judge's verdict is only as good as the reference. Nothing in the reference-anchoring literature tests what happens when the reference itself is corrupted.

Relation to Prior Work

Four systems sit closest to this proposal, and each makes an assumption we drop.

SPARE aligns each generated step to a reference solution in a single pass, using matching rules that allow one step to map to several. It assumes the reference is complete and gold. We assume neither.

RetrievalPRM retrieves similar solved problems and steps from PRM800K and Math-Shepherd and uses them as in-context examples for a process reward model, which is a model that scores each step of a solution. It reaches 65.8 average F1 on ProcessBench. Its retrieved items are curated, trusted exemplars of how to judge. Ours are untrusted factual anchors for what is true.

R3-RAG rewards each retrieval step using an LLM critic that judges document relevance with no reference at all. That is exactly the kind of judge our diagnostic predicts will be captured.

ClueAnchor trains the generator to extract a supporting clue before reasoning, using the gold answer as a training-time guide. It gives no inference-time check of whether a chosen clue is itself valid.

RevisEval does not supply the method but it constrains the design. It builds references by revising the candidate response. For reasoning steps, that risks laundering a flawed step into its own reference, so our anchors are always retrieved independently of the candidate.

CofCA motivates the evaluation. It shows GPT-4's multi-hop reasoning chains are fully correct only 36.3% of the time, even though final-answer exact match on factual data is around 70%. CofCA diagnoses the problem but offers no verifier.

Research Program

The program has three parts: a diagnostic study, a verifier, and a downstream test.

Part one is the diagnostic study of anchor capture. We take judging tasks that already come with references. These are VerifyBench and VerifyBench-Hard, plus the ProcessBench tiers whose source problems carry reference solutions. We then corrupt the references in controlled ways: a changed number, a swapped entity, a dropped or inverted step.

For each judge, we compare three conditions: accuracy with the clean reference, with no reference, and with the corrupted reference. We define the capture rate as the fraction of items where the judge was right without a reference but becomes wrong with the corrupted one. The judges tested are Qwen2.5-7B/72B-Instruct, a thinking-mode model such as QwQ-32B, Prometheus-style rubric judges (Prometheus), and Universal-PRM-7B (AURORA).

We then repeat the study in the RAG setting. Here the "reference" is a retrieved passage, and we control corruption with a rate $\rho$, the fraction of anchor passages replaced by a CofCA-style counterfactual or a SciAux misleading snippet. This part stands on its own. If capture is negligible, the rest of the program is unnecessary, and saying so would be the result.

Part two is the anchor-skeptical verifier. For each step in a candidate reasoning chain, the step is turned into a claim query. We retrieve $k$ anchors from the task's designated corpus using queries issued independently of the generator. The verifier is deliberately not shown the generator's own retrieved context.

A 7B verifier, initialized from Qwen2.5-7B-Instruct, labels the step against each anchor. It then produces one of three outputs: supported, contradicted, or anchors-unreliable. The last output is the key addition. It lets the verifier abstain on factual grounding and fall back to a logic-only check.

Training data is built from the training splits of HotpotQA, 2WikiMultiHopQA, and MuSiQue. We rewrite them counterfactually in the CofCA manner so that parametric memory cannot shortcut the check. Each question comes with gold sub-question answers, which give exact step labels. We then inject corrupted anchors at known positions, so the correct three-way label is known by construction. CofCA itself and SciAux are held out entirely for evaluation.

Part three is the downstream test. The verifier's step scores are used for best-of-$n$ chain selection and for early stopping, on retrieval-augmented reasoning chains generated by a fixed policy model.

Baselines

The naive reference point is outcome-only selection. Chains are chosen by majority vote over final answers, and graded by final-answer exact match. This is the "do nothing" option that the 51.8% hidden-error rate on Omni-MATH (ProcessBench) and CofCA's 36.3% chain correctness argue against. We run it on the same sampled chains, with the same $n$.

The strongest general step judge is an unanchored thinking model. On ProcessBench, o1-mini reports 87.9 average F1 and QwQ-32B-Preview reports 71.5 (ProcessBench). We run the best open thinking model we can afford, zero-shot, on exactly the same chains and steps. Beating it means higher step-error F1 on the RAG benchmarks at every $\rho > 0$, while matching it at $\rho = 0$.

The strongest trained step verifier is Universal-PRM-7B, which reports 74.3 ProcessBench F1 and 74.5 on UniversalBench (AURORA). The strongest retrieval-augmented verifier is RetrievalPRM-7B, at 65.8 average F1 and 57.33 on Omni-MATH (RetrievalPRM). On math we compare against their reported numbers directly. On the RAG benchmarks we rerun them at 7B scale. RetrievalPRM will be given our anchor pool in place of its exemplar pool, which keeps retrieval budget and backbone size matched.

The shared-context judge is the most important competitor. It is the same 7B backbone, fine-tuned on the same data, but it verifies each step against the documents the generator retrieved. This is essentially an anchored version of the R3-RAG critic. It isolates whether independent retrieval matters.

Two ablations isolate the key ingredients. The first removes corrupted anchors from training. The second forces a binary output with no anchors-unreliable option. The proposal counts as successful if both ablations show higher capture rates at $\rho \geq 0.3$ than the full verifier.

For the RAG benchmarks, no reported result was found on Lacuna for step-level verification F1 on CofCA or SciAux. We will establish those numbers ourselves for all baselines under a shared tuning budget: the same number of training examples, epochs, and hyperparameter trials per trained model.

Why not just…?

Why not just use a strong thinking model as an unanchored judge? On clean math it is strong: o1-mini reaches 87.9 F1 on ProcessBench. However, reference withholding costs frontier judges 5% to 18% on VerifyBench, and unanchored judges fail exactly where their parametric knowledge is missing (Doddapaneni et al.). CofCA's counterfactual passages are built to create that situation. The real threat is that a thinking judge might be less captured than an anchored one, because it ignores the anchors. Our $\rho$ sweep settles this directly. If the unanchored thinking judge beats our verifier at moderate corruption, anchoring is not worth it in RAG, and we will report that.

Why not just give the verifier the generator's retrieved documents? This is the cheapest anchored design. It is also the one most likely to share the generator's mistake. If a misleading snippet caused the error, the same snippet will endorse it. SciAux shows reasoning chains rationalize such snippets rather than reject them. The shared-context baseline tests this directly. If it matches independent retrieval, we drop the independence requirement and keep the rest.

Why not just filter or rerank documents before generation? Filtering reduces noise, but it cannot catch errors that arise in the reasoning itself. Those errors dominate CofCA, where chains fail through broken intermediate logic even with the right passages. A reranker also has none of the logical capacity that R3-RAG identifies as the retriever bottleneck. We will add a strong reranker in front of every baseline, so the comparison measures what step checking adds beyond cleaner context.

Why not just take a majority vote over several retrieved anchors, with no special training? This is a real threat. With independent anchors and low $\rho$, voting may recover most of the gain. It is precisely the "no corrupted-anchor training" ablation run with multiple anchors. If it matches the full verifier across $\rho$, the contribution reduces to the diagnostic plus independent multi-anchor retrieval, and we would say so. We expect voting to break when corrupted anchors are correlated, as when several passages repeat the same counterfactual. We will include that case explicitly.

Proposed Experiments

The decisive check comes first and is cheap. On VerifyBench-Hard, where the best system reports 87.9% and frontier models sit around 70-75% even with clean references, we measure accuracy under clean, withheld, and corrupted references. Anchor capture is real if the corrupted-reference accuracy falls below the withheld-reference accuracy for most judges.

We repeat the same three-way comparison on ProcessBench items that carry references, and on CofCA and SciAux with retrieved anchors across $\rho \in \{0, 0.1, 0.3, 0.5\}$.

The main experiment trains the anchor-skeptical verifier and reports four quantities across the $\rho$ sweep, against the baselines above: step-error F1 (in the ProcessBench style of locating the earliest wrong step), capture rate, and abstention precision (how often "anchors-unreliable" coincides with an actual corrupted anchor).

The downstream experiment uses the verifier for best-of-$n$ selection on retrieval-augmented chains. It reports CofCA exact match and full-chain correctness, against GPT-4's reported 36.3% chain correctness as a reference point. It also reports SciAux accuracy under two misleading snippets, where the target is to recover a meaningful part of the gap between 77.03% and the clean 96.22% reported for Qwen3-8B. Finally, a transfer check runs the RAG-trained verifier, unchanged, on ProcessBench's reference-carrying tiers. This tests whether skepticism toward anchors learned on text carries over to math references.