Systems that answer multi-hop questions, meaning questions that require chaining facts across several documents, are increasingly trained on reasoning traces that a large language model (LLM) has generated and that a second LLM acting as a judge has screened. Both halves of that pipeline are weak in documented ways. The search that generates the traces tends to collapse onto one retrieval route per question. The judge that screens them favors fluent, familiar-looking text while giving no sign that it is doing so. We propose replacing the judge's opinion of whether a trace is faithful to its evidence with a structural test. An answer counts as trustworthy when the search reaches it through two or more chains of retrieved documents that overlap little. The search itself is rewarded for finding such alternative chains. The two pieces support each other: the diversity reward targets evidence routes rather than wording, so it works against mode collapse, and agreement across routes with little shared evidence gives a hallucination check that never asks a model to read and grade prose.
In two sentences: run tree search over retrieval-and-reasoning steps, rewarding paths that use different documents from the paths already found. Then keep for distillation only the answers that independent evidence chains confirm, rather than the traces an LLM judge rates as faithful. Our claim is that this filter keeps fewer hallucinated intermediate steps than judge-based filtering on 2WikiMultiHopQA and MuSiQue, that students trained on its output are more accurate and less collapsed, and that the gain comes specifically from provenance diversity. It should not be reproducible by simply sampling more chains and taking a majority vote.
Core Claim
The closest existing system is PathFinder. It runs Monte Carlo tree search (MCTS) offline over question decompositions. It then filters traces in two stages: first by recall of intermediate sub-answers against dataset annotations, then with a Qwen-72B judge that scores logical consistency and faithfulness to retrieved context. The surviving traces are distilled into 7B to 9B students. PathFinder treats the judge as a reliable signal, and its search optimizes for finding one good path per question.
Our proposal differs in two ways. First, the search objective explicitly rewards evidence-path diversity. Second, the faithfulness test is defined over document provenance, not judged text. The sub-answer recall test needs annotated intermediate answers, so it does not exist outside annotated training sets. The judge test is the part that would carry over to unannotated corpora, and it is exactly the part we argue is compromised.
The motivation for distrusting that test is direct. The Silent Judge shows GPT-4o's pairwise verdicts shift by up to +30% on ELI5 in response to arbitrary timestamps, and it shows that judges acknowledge such cues 0% of the time. BiasScope finds exact-match bias, which rewards outputs that mirror the prompt's wording, and reports a 74.7% GPT-4o error rate on JudgeBench-Pro. Local Coherence or Global Validity? finds a GPT-4o judge only 57.8% accurate at tagging step errors in polished but invalid math traces. None of these studies looks at multi-document attribution. We expect the same failures to matter more there, because a judge can easily accept a fluent bridge entity that no retrieved document actually states.
We define provenance concretely. Each path $p$ from the root to an answer carries an answer $a(p)$ and the set $D(p)$ of documents it actually read. Two paths are independent to the degree their evidence differs, measured by $d(p,q) = 1 - |D(p)\cap D(q)| / |D(p)\cup D(q)|$, one minus the Jaccard overlap of their document sets. The corroboration score of a candidate answer $a$ is
$$C(a) = \max_{p,q:\ a(p)=a(q)=a} d(p,q),$$
that is, the evidence separation of the most independent pair of paths agreeing on $a$. An answer reached only through a single shared chain scores zero. During search, a node that completes a path receives its usual process reward minus $\lambda$ times its maximum Jaccard overlap with completed paths already in the tree. This is the similarity-penalty form common in the mode-collapse literature, but here the penalty is applied to document sets rather than strings or hidden states.
Relation to Prior Work
Two lines of exploration work supply the reward shape and also bound its scope. Solution Divergence scores diversity from a graph Laplacian built on edit distance between solutions and reports $R^2$ up to 0.94 between divergence and Pass@1. Its authors note that edit distance captures surface wording, not inferential structure. RepExp rewards correct answers whose hidden-state representations are novel. It reports a 13.4x sample-efficiency gain over group-relative policy optimization (GRPO), a common reinforcement-learning method for LLMs, on GSM8K, at roughly 37% extra training wall-clock time. Both were tested only where an automatic verifier exists. Neither tells us whether pushing for novelty in an open corpus finds real alternative routes or invents bridge entities. We reuse their penalty form but replace the similarity measure with document provenance. That replacement is the ingredient our ablations isolate.
On the Limits of Language Generation proves that, in its formal model, no generator can achieve both consistency (no hallucination) and breadth (no mode collapse) in general. It names as open problems mapping the trade-off between approximate breadth and consistency, and building feedback loops that balance the two. Our setting is far from its formal model, so we do not claim to test the theorem. We use it as framing: corroboration restricts breadth to evidence-supported routes, and we measure the empirical trade-off between breadth and faithfulness directly.
Three multi-hop systems constrain the design. GRAT supplies the MCTS-with-process-rewards machinery we build on. Before Reasoning Can Fail warns that agents often answer without reading documents, and that this worsens with more internal reasoning. A novelty reward could make that worse, because the cheapest way to look novel is to skip retrieval. For that reason $D(p)$ counts only documents that were actually read, and the zero-read rate is a tracked outcome. AnchorCoT shows that committing to an ordered chain of bridge entities stabilizes generation. Its single deterministic anchor sequence is the opposite of what we want, but its anchor extraction gives us a second, entity-level independence measure to report alongside document overlap.
Research Program
The first step is a feasibility analysis. Corroboration only helps if multi-hop questions often admit more than one evidence chain. MuSiQue was built by composing single-hop questions specifically to block shortcuts, so on that benchmark alternative chains may be rare. Using GRAT-style search with a Llama3-8B or Qwen-7B policy over the standard Wikipedia corpora, we will measure the fraction of questions where at least two correct paths have $d(p,q) \ge 0.5$. We will report this separately for 2WikiMultiHopQA, HotpotQA, and MuSiQue. Questions without an alternative chain get no corroboration score, so the method abstains on them. Coverage is therefore a first-class result. If coverage is near zero on MuSiQue, the method is honestly limited to redundant corpora, and we will say so.
The second step is a bias audit of judge filtering in the multi-hop setting. We take MCTS traces from 2WikiMultiHopQA, where gold supporting facts and intermediate answers give ground truth for whether each hop is faithful. We then apply perturbations adapted from Silent Judge and BiasScope: provenance labels on documents, phrasing that mirrors the question, and a fluent but unsupported bridge-entity claim inserted into an otherwise correct trace. For each perturbation we measure how often a Qwen-72B faithfulness judge, configured as in PathFinder, accepts traces containing a fabricated hop. We compare that against how often the corroboration filter accepts the same answers. This step establishes whether the problem we claim actually exists in multi-hop distillation.
The third step builds the full pipeline. We run provenance-diverse search, filter by corroboration (keeping answers with $C(a)$ above a threshold tuned on a held-out split), and distill the kept traces into a 7B student using the same supervised fine-tuning recipe as PathFinder. We then evaluate the student's accuracy, its rate of hallucinated intermediate steps, and its own evidence-path diversity at test time. The test-time diversity check asks whether filtering on corroboration also prevents the student from collapsing onto one retrieval template.
Baselines
The naive reference point is a single interleaved retrieve-and-reason chain in the style of IRCoT, run on the same policy model, retriever, and corpus. It shows what the search and filtering add at all.
The strongest existing method for this question is PathFinder. It is the only published pipeline that combines tree search, LLM-judge faithfulness filtering, and distillation for multi-hop questions, so it is precisely the design we claim to improve. PathFinder reports an F1 of 78.38% on 2WikiMultiHopQA with a Qwen-7B student, against 73.48% for DeepRAG. We found no reported PathFinder number on Lacuna for MuSiQue or HotpotQA.
We will reproduce PathFinder ourselves rather than compare against its published figure. Both pipelines will use the same search budget (same number of rollouts per question), the same Qwen-72B teacher, the same student, and the same number of distilled trajectories. We will match the count of kept examples so that our filter cannot win simply by keeping more or less data.
Beating PathFinder means equal or higher student F1 on 2WikiMultiHopQA together with a significantly lower rate of hallucinated hops in the student's traces, where a hop counts as hallucinated if it is not supported by gold supporting facts. Lower hallucination at equal F1 already counts as a win, because faithfulness is the claim.
GRAT is the second strong comparison. It reports 75.2% accuracy on 2WikiMultiHopQA with Llama3-70B, and an improvement of Llama3-8B from 60.6% to 66.8% after fine-tuning on 11,000 search-generated trajectories. We will match that 11,000-trajectory budget when using an 8B student.
Self-consistency is the most dangerous baseline: sample many chains and keep the majority answer. We run it with the same total number of sampled paths as our search. Majority agreement serves as its filter, with the threshold tuned on the same held-out split.
Three ablations isolate the key ingredient. The first keeps the provenance diversity bonus but filters with the judge. The second keeps corroboration filtering but drops the diversity bonus. The third replaces document Jaccard distance with edit distance in the style of Solution Divergence and with a RepExp-style representation bonus, while holding everything else fixed.
READ-GATE reports net gains of 3.2 to 9.4 points from refusing answers when nothing was read. It addresses a different failure, so we apply it to every system as a common layer rather than treating it as a competitor.
Why not just…?
Why not just use self-consistency voting instead of provenance corroboration? This is the most serious threat. If many sampled chains agree, that may already be a good signal. The case against it is that sampled chains for the same question usually issue similar queries and read the same top documents. Their agreement can therefore reproduce one shared retrieval error, or one shared guess from the model's own memory, many times over. The comparison is direct: at matched sampling budget, we compare the hallucinated-hop rate among accepted answers for majority agreement and for $C(a)$. We also stratify by the evidence overlap of the agreeing paths. If voting matches corroboration even among high-overlap agreements, the provenance component adds nothing. In that case we would report the null result and drop it.
Why not just debias the judge with order swapping and an instruction to ignore metadata? The Silent Judge result explains why verbal instructions are unreliable: judges never mention the cues they use, so there is no visible behavior to correct. The audit in our second step will nevertheless include a prompt-debiased judge as an extra arm, because whether prompting fails for multi-document faithfulness specifically has not been measured.
Why not just raise sampling temperature to get diverse paths? Token-level randomness diversifies wording far more than it diversifies document routes, and in multi-hop chains an early wrong entity compounds through every later hop. We will report evidence-path diversity, using the provenance-graph analogue of the Solution Divergence spectral score, for temperature sampling at matched budget. If temperature sampling yields as many low-overlap correct paths as the diversity bonus does, the bonus is unnecessary and only the filter matters. The second ablation tests this.
Why not just filter on gold sub-answer recall, as PathFinder's first stage does? Where annotations exist, that filter is strong, and we use it as the upper bound in the audit. It cannot be applied to new corpora or to unannotated training questions, and that is exactly the situation in which pipelines fall back on the judge.
Decisive Experiments
Three results decide the proposal.
The first comes from the bias audit on 2WikiMultiHopQA. Among traces containing an injected unsupported bridge claim, the corroboration filter should accept a substantially smaller fraction than the Qwen-72B judge does. Its acceptance rate should also stay stable when the stylistic and provenance cues are added, whereas the judge's acceptance should shift.
The second compares students at matched data count on 2WikiMultiHopQA, plus MuSiQue where coverage permits. The corroboration-filtered student should match or exceed the reproduced PathFinder F1 while showing fewer hallucinated hops, more distinct evidence routes at test time, and no increase in zero-read answers.
The third tests whether the provenance measure is what matters. The edit-distance and representation-distance variants should fall short on hallucinated-hop rate at matched diversity, and self-consistency should fall short at matched budget. Failure of the first result means LLM judges are not the bottleneck in multi-hop distillation. Failure of the third means a simpler agreement signal suffices. Either finding would be worth reporting.
References
- PathFinder: MCTS and LLM Feedback-based Path Selection for Multi-Hop Question Answering
- The Silent Judge: Unacknowledged Shortcut Bias in LLM-as-a-Judge
- BiasScope: Towards Automated Detection of Bias in LLM-as-a-Judge Evaluation
- Local Coherence or Global Validity? Investigating RLVR Traces in Math Domains
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Representation-Based Exploration for Language Models: From Test-Time to Post-Training
- On the Limits of Language Generation: Trade-Offs Between Hallucination and Mode Collapse
- GRAT: Guiding Retrieval-Augmented Reasoning through Process Rewards Tree Search
- Before Reasoning Can Fail: Pre-Evidence Procedural Failures in Agentic RAG
- AnchorCoT: Anchors Pave the Way for Multi-hop Reasoning
- Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions