A classic failure in text-coherence research is the "word salad": a passage where every sentence connects smoothly to the next, through a shared word or a transition like "therefore," yet the passage as a whole argues nothing. Models that only judge adjacent sentence pairs are easily fooled by it (Farag et al., 2018). We argue that multi-hop question answering has the same failure, but for evidence rather than prose. A system that retrieves documents step by step, where each new document shares an entity with the previous one, can assemble a chain in which every link looks plausible and the chain still does not answer the question. One hop may be a topical distractor that mentions the bridge entity under the wrong relation. The chain may end on the wrong attribute. A step may be skipped entirely. Current retrievers and rerankers score documents one at a time, or score adjacent pairs, so they cannot see this kind of error.
We propose to train a small verifier that scores an ordered evidence chain against the question as a whole. It is trained with synthetic corruptions that keep every adjacent link intact but break the chain's overall logic. We then use it inside an iterative retrieval loop to decide when to stop, and when to throw away a hop and search again. The two-sentence version is this: multi-hop retrieval errors are locally coherent but globally broken, so we train a detector on exactly that kind of negative example and let it trigger backtracking. The test is whether this beats rerankers and LLM self-checks that were trained or prompted on ordinary negatives.
Core Claim
The coherence literature showed that local transition models miss document-level organization. It also showed that the fix is training signal at the level of large structural units. In Corruption Is Not All Bad, pre-training an essay encoder to detect which corruption had been applied reached 0.155 MSE on ICLE organization scoring. In that work, paragraph-level shuffling and same-prompt paragraph replacement helped much more than sentence-level shuffling. Our claim is that the same lesson transfers to multi-hop evidence. The unit is a hop rather than a paragraph. The useful corruptions are the ones that preserve local entity links while breaking the global relation path.
We expect a verifier trained on these "locally valid, globally broken" negatives to catch substantially more bad chains than one trained on random or merely irrelevant negatives. We also expect the gains to grow with hop count, because the number of places where a locally plausible substitution can occur grows with chain length. If the hard-negative verifier does no better than a reranker trained on the same gold chains with ordinary negatives, the claim is false.
This differs from the closest existing work in specific ways. LOG filters documents by their individual usefulness and then clusters the whole evidence set together with a multi-positive contrastive loss. It treats the chain as an unordered bag and never trains against negatives that share bridge entities by construction. PAGE-RAG names the underlying problem, that connectivity is not support, and penalizes high-degree hub entities with a heuristic path-coherence score. However, it works only as a post-retrieval selector over a fixed top-20 pool, so it cannot recover a hop that pool misses. GlobalRAG fixes a question-decomposition plan upfront and rewards faithful execution with reinforcement learning. Its authors list the lack of re-planning when evidence contradicts the plan as a limitation. Our verifier is learned, sensitive to hop order and relation, and consulted during retrieval, so a low chain score can trigger a new search rather than only a reranking of what has already been found.
Research Program
The first component is a corruption generator for evidence chains. We build it on MuSiQue, whose questions are composed bottom-up from single-hop questions with known gold paragraphs and known reasoning-graph shapes. That construction lets us edit one hop at a time. We define four hop-level corruption families.
Bridge-preserving substitution replaces one gold paragraph with a distractor that mentions the same bridge entity but supports a different relation. For example, a passage about Nolan's filmography replaces one about his birthplace. Terminal relation swap keeps the path but ends on a paragraph giving the wrong attribute of the correct final entity. Hop deletion removes an intermediate paragraph, which produces the disconnected reasoning MuSiQue's filtering was designed to expose. Order inversion permutes hops in chains whose dependency structure is linear.
For each corrupted chain we measure a local score: the minimum over adjacent paragraph pairs of an off-the-shelf pairwise relevance model. We keep only corruptions whose local score stays within the range of gold chains. This filter enforces the word-salad property, so the negatives cannot be caught by any adjacent-pair check.
The second component is the verifier itself. It is a cross-encoder over the question plus the ordered chain, trained as a multi-class classifier that predicts which corruption, if any, was applied. This is the formulation of Corruption Is Not All Bad, and the auxiliary labels give denser signal than a binary valid/invalid target. Its output is a chain score $S(q, c)$ for question $q$ and ordered chain $c$. We quantify the gap it targets as
$$G = P\big(L(c) \ge \tau,\ c \text{ invalid}\big),$$
where $L(c)$ is the minimum adjacent-pair score and $\tau$ is the threshold at which a local scorer accepts 95% of gold chains. $G$ is the rate at which invalid chains pass every local check. We report it for existing systems' actual retrieved chains, not just for our synthetic negatives.
The third component is a controller that runs inside an iterative retriever in the style of IRCoT. After each retrieval step it scores the current chain prefix. The controller has three actions. It stops if $S$ exceeds a threshold. It continues if $S$ is flat. It backtracks if adding the newest hop lowers $S$ by more than a margin: that hop is dropped, and the step's query is regenerated with the rejected paragraph excluded. Backtracking targets the error accumulation that Wang et al. (2025) and LOG describe, where one misleading early hop corrupts every later query. Unlike GlobalRAG's static plan, the controller needs no upfront decomposition.
Baselines
The naive reference point is single-step dense retrieval followed by the same reader. We also run plain IRCoT with its native stopping rule, on the same retriever, reader, and corpus. This measures what iteration alone buys. The file gives IRCoT's gains as 11–21 recall points and up to 15 QA F1 points over one-step retrieval. It gives no absolute IRCoT scores on MuSiQue in a form we can quote, so we will reproduce IRCoT ourselves. DualRAG reports that IRCoT uses 3.76–4.26 retrieval iterations per HotpotQA question, which gives a reference point for our cost comparison.
The strongest learned competitor is LOG. It is the closest in spirit and reports 53.3 Answer F1 and 78.6 Support F1 on MuSiQue-Ans. It is also the published method that most directly optimizes whole-chain evidence quality, which is our question. We will run LOG's released procedure on the same backbone and training split as our verifier. Beating it means exceeding 53.3 Answer F1 on MuSiQue-Ans at equal reader size, or matching it while showing a clear Support F1 gain on 4-hop questions, where LOG reports its own largest improvements (8.6% retrieval, 6.5% QA).
The strongest selection-layer competitor is PAGE-RAG. It reports +10.4 Support F1 and +3.3 Answer F1 over NV-Embed-v2, averaged across HotpotQA, MuSiQue, and 2WikiMultiHopQA, and it raises SelfRAG's Answer F1 from 62.39 to 69.36. We run it on the same top-20 pools our controller can draw on, with the same token budget. This isolates whether learned chain scoring plus backtracking beats heuristic path scoring plus minimal selection.
The key ablation is our verifier trained on exactly the same gold chains and the same number of negatives, but with the negatives drawn as random or BM25-retrieved non-gold paragraphs. The locality-preserving filter is off in this condition. This isolates the one ingredient we claim matters: negatives that are locally coherent and globally broken. A second ablation keeps the verifier but removes backtracking, using $S$ only as a final reranker. This separates the value of the detector from the value of acting on it during retrieval. All learned baselines get the same hyperparameter-search budget, measured in GPU-hours, and the same reader.
Why not just…?
Why not just fine-tune a standard cross-encoder reranker on MuSiQue's gold supporting paragraphs? This is the most serious threat, and our key ablation is exactly this system. Our reason to expect it to fall short is that ordinary negatives are mostly off-topic. An ordinary reranker can therefore learn entity overlap as a proxy for support, which PAGE-RAG shows is the dominant confusion. The word-salad result of Farag et al., 2018 is the coherence-side evidence that such proxies break under adversarial local smoothness. If the ablation matches the full verifier on the gap measure $G$ computed over real IRCoT chains, we will conclude that hard negatives add nothing and report it.
Why not just ask the reader LLM to check its own evidence, as NeuroPath's reflection step does? NeuroPath uses an LLM to prune graph branches and then reflects after retrieval to fill gaps. It reports 16.3% Recall@2 gains over HippoRAG and LightRAG, so LLM self-checking clearly helps. It also costs extra full-model calls per hop. The question is what a small trained verifier achieves per unit of compute. We will run an LLM-judge controller with the same stop/continue/backtrack actions, prompted with the corruption taxonomy. We compare at matched total FLOPs and matched wall-clock time. If the LLM judge wins at equal cost, the practical contribution shrinks to the diagnostic benchmark.
Why not just retrieve more documents and let a long-context model sort it out? PAGE-RAG's own experiments show that widening the pool from top-5 to top-20 without structural filtering adds topical distractors. They also show that better selection within a fixed budget improves both Answer and Support F1. Bridge-preserving substitutions are exactly the distractors a larger pool admits. We include a top-50 flat-context condition to measure directly whether its error rate on our corrupted evaluation chains is lower or higher than the verifier's.
Proposed Experiments
The first experiment is diagnostic and needs no new model. We collect chains actually produced by IRCoT, ITER-RETGEN, and PAGE-RAG on MuSiQue-Ans and 2WikiMultiHopQA. We label each chain as valid or invalid against the gold paragraphs, then compute $G$. If few invalid chains pass local checks, the premise is wrong and the project stops here.
The second experiment trains the verifier. We evaluate it on held-out corrupted chains, broken down by corruption family and hop count, against the random-negative ablation.
The third experiment runs end-to-end question answering. We evaluate on MuSiQue-Ans, and on MuSiQue-Full, whose unanswerable contrast questions test whether a low chain score correctly leads to abstention. We also evaluate on 2WikiMultiHopQA and HotpotQA, and on Bamboogle as an out-of-domain check the verifier never trained on. We report Answer F1, Support F1, and EM. For reference, MuSiQue humans reach 78.0 Answer F1 and 93.9 Support F1. We also report retrieval iterations per question and wall-clock latency.
Two results would decide the question. The first is whether the hard-negative verifier beats the random-negative ablation on 3- and 4-hop MuSiQue questions by a margin larger than seed variance. The second is whether backtracking beats verifier-as-reranker when the needed hop lies outside the initial top-20 pool. That case is precisely what PAGE-RAG cannot recover, so it is where acting during retrieval should matter. If both hold and the gain survives on Bamboogle, the local-global framing has paid for itself.
References
- Neural Automated Essay Scoring and Coherence Modeling for Adversarially Crafted Input
- Corruption Is Not All Bad: Incorporating Discourse Structure Into Pre-Training via Corruption for Essay Scoring
- LOG: A Local-to-Global Optimization Approach for Retrieval-based Explainable Multi-Hop Question Answering
- PAGE-RAG: Provenance-Aware Graph Evidence Promotion for Fixed-Budget Multi-hop Retrieval-Augmented Generation
- GlobalRAG: Enhancing Global Reasoning in Multi-hop Question Answering via Reinforcement Learning
- ♫ MuSiQue: Multihop Questions via Single-hop Question Composition
- Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions
- DualRAG: A Dual-Process Approach to Integrate Reasoning and Retrieval for Multi-Hop Question Answering
- NeuroPath: Neurobiology-Inspired Path Tracking and Reflection for Semantically Coherent Retrieval
- Enhancing Retrieval-Augmented Large Language Models with Iterative Retrieval-Generation Synergy