Multi-hop question answering systems answer a question like "where was the director of Inception born?" by looping: they retrieve a document, write an intermediate answer, use it to retrieve the next document, and so on. Every loop adds more text to the model's context, and some of that text is irrelevant. We already know that irrelevant text hurts language models on single-step questions. What we do not know is whether one bad hop hurts the next hop even after the retriever brings back the right evidence. If it does, the failure is a persistent state the chain falls into, not an isolated mistake. This proposal tests that hypothesis directly and builds a controller that responds to it. When the controller detects that the chain has entered such a state, it rolls the context back to the last hop that was grounded in evidence. Existing controllers only decide whether to stop.
The approach borrows a simple tool from sequence modeling. We describe each hop as being in one of two states, grounded or ungrounded, and estimate the probabilities of moving between them. This is a first-order Markov chain, meaning a model in which the next state depends only on the current one. The question is whether "ungrounded" is sticky. The key methodological step is to measure that stickiness while holding the quality of the next retrieval fixed. We do this by injecting distractors deliberately. That turns a correlational diagnostic into a causal one.
Core Claim
We claim three things. First, in iterative retrieval-augmented generation (RAG, where a model answers from documents fetched by a search step), an ungrounded hop raises the probability that the next hop is ungrounded. This effect holds even when the next hop receives the gold supporting paragraph. Second, this persistence can be read off the reader model's hidden states before the next hop is generated. Third, a policy that can roll back contaminated context beats policies that can only stop or continue.
Define $g_t \in \{0,1\}$ as whether hop $t$'s sub-answer is grounded. Grounded means it matches the gold sub-answer and cites a gold supporting paragraph. Define $r_{t+1} \in \{0,1\}$ as whether the gold paragraph for hop $t+1$ is present in that hop's retrieved set. The quantity of interest is the conditional trapping gap
$$\Delta = P(g_{t+1}=0 \mid g_t=0,\ r_{t+1}=1) - P(g_{t+1}=0 \mid g_t=1,\ r_{t+1}=1).$$
If $\Delta \approx 0$, errors propagate only through bad retrieval, and cleaning each hop's evidence is enough. If $\Delta > 0$, the context itself carries the failure forward, and the chain must be edited.
Relation to Prior Work
Three lines of work supply the ingredients, but none measures persistence inside retrieval chains.
The first line motivates the trapping idea. Old Habits Die Hard fits exactly this kind of two-state chain to multi-turn conversations. It finds transition-matrix traces above 1 (average 1.31), meaning states tend to persist. It also finds that the chain's persistence correlates with an angular separation in hidden states at about 85% depth (Spearman 0.78). However, it studies conversational history with no retrieval, and it only measures persistence without intervening. We move the measurement into retrieval chains. We also add the conditioning on $r_{t+1}$ that separates contaminated context from bad retrieval. Their finding that the effect collapses under shuffled, topically incoherent sequences is a constraint on our design. Multi-hop chains are topically coherent by construction, so we expect persistence to appear, but it may be weaker when distractors are off-topic.
The second line supplies the control method. Stop-RAG casts iterative RAG as a Markov decision process, a formal description of sequential choices, with only STOP and CONTINUE actions. It trains an offline Q-network using $Q(\lambda)$ targets and reaches 47.0 F1 on MuSiQue, against 44.8 for letting the language model decide when to stop. We reuse its training recipe and add a third action, ROLLBACK. Stop-RAG treats the generator's intermediate state as a black box. It also underperformed on HotpotQA because its generator had been fine-tuned for a fixed iteration count. We test whether that coupling problem shrinks when the policy can repair context rather than only truncate it. Incremental Sequence Classification with Temporal Consistency supplies a second tool: a temporal-consistency loss (TC-λ) for scoring partial sequences. We use it to train the probe that estimates grounding from each prefix of the chain.
The third line constrains which fixes are worth trying. Breaking Focus shows that coherent distractors cause roughly a 45% accuracy collapse and that telling the model to ignore irrelevant text does not help. DAS and LDAR filter distractors before reading. LDAR also documents an inverted-U curve: accuracy first rises, then falls, as more context is retrieved. Both methods operate on a single retrieval step. BRIEF compresses evidence in one pass. Our contribution is not a better filter. We ask whether filtering at each hop is enough, and the conditional gap $\Delta$ answers exactly that. CoRAG observes that irrelevant intermediate retrievals derail later hops but does not separate the two possible causes.
Research Program
The work proceeds in three stages.
Measurement stage. We use CoRAG-style chains on a Llama-3.1-8B-Instruct generator over MuSiQue. MuSiQue's gold single-hop decompositions and supporting paragraphs let us label $g_t$ and $r_t$ automatically. We run chains naturally to estimate the transition matrix and $\Delta$ observationally.
We then run the causal version. At hop $k$, we replace the gold paragraph with a distractor. We use two kinds: a retrieved hard negative, and an adversarial passage in the style of the error-guided perturbations in Breaking Focus. At hop $k+1$, we force the gold paragraph into the retrieved set. Comparing against matched chains without injection gives an interventional estimate of $\Delta$. We report it by hop depth, distractor type, and accumulated context length. The length breakdown connects persistence to LDAR's inverted-U curve.
Prediction stage. We train a lightweight probe on hidden states across layers to predict $g_{t+1}$ before hop $t+1$ is generated, using TC-λ targets. We compare it against the model's entropy over its answer and against an angular-separation statistic modeled on Old Habits Die Hard.
Control stage. We extend Stop-RAG's offline $Q(\lambda)$ policy to three actions. ROLLBACK deletes all context added since the last hop the probe judged grounded. It then reissues that hop's sub-query with the contaminating passages excluded. We cap rollbacks per question so the token budget stays comparable.
Baselines
The do-nothing reference is a fixed-iteration IRCoT-style loop. IRCoT generates a reasoning sentence and uses it as the next retrieval query. We run it with the same generator and retriever and with an iteration count tuned on development data. It anchors how much any control adds.
The strongest existing competitor is Stop-RAG applied to the same CoRAG-style generator. Its reported number is 47.0 F1 on MuSiQue (Stop-RAG). We will retrain it ourselves on identical offline trajectories, with the same Q-network capacity and the same hyperparameter search budget as our three-action policy. The only difference between the two systems is then the ROLLBACK action and the probe features. Beating it means a higher F1 on the MuSiQue test set at equal or lower average retrieved tokens, with the difference surviving a paired bootstrap across three seeds. We also require that the gain concentrates on questions where the probe fired. Otherwise the improvement is not coming from the proposed mechanism. For graph-based retrieval we report HopRAG's published 42.20 EM on MuSiQue as context. It uses different indexing, so we do not treat it as a matched comparison.
Three ablations isolate the key ingredient. The first applies DAS-style filtering by the probability of the query given the document at every hop, with no rollback. The second is a stateless loop that passes only sub-answers forward and never accumulates passages. The third triggers rollback randomly at the probe's firing rate, so any gain from rollback-as-regularization is not credited to detection.
Why not just…?
Why not just tell the model to ignore irrelevant passages? Breaking Focus reports that this instruction fails against coherent distractors. We include it as a prompt variant of the do-nothing baseline anyway, because it costs nothing to run.
Why not just filter evidence at each hop with DAS or LDAR? This is the real threat. If $\Delta$ is near zero once $r_{t+1}=1$, bad hops only hurt through bad retrieval, and per-hop filtering is the right fix. In that case this proposal's controller is unnecessary. The interventional measurement settles this before any controller is built. If $\Delta$ is small, we report that result and stop at the per-hop filtering baseline rather than argue for rollback.
Why not just never accumulate context, passing only sub-answers between hops? That removes contaminated passages. However, it also discards the evidence trail that IRCoT credits with cutting factual errors by up to 50%, and it cannot remove a wrong sub-answer that is itself the contamination. The stateless ablation tests this directly. If it matches our rollback controller, the simpler design wins.
Why not just give Stop-RAG more training data? A policy that can only stop cannot recover a question whose chain went wrong early. It can only truncate it. The HotpotQA shortfall reported in Stop-RAG is where this limitation should show. We report per-dataset results to check it.
Decisive Experiments
The first decisive check is the interventional $\Delta$ on MuSiQue with 95% confidence intervals. We expect it to be clearly positive for coherent distractors and near zero for off-topic ones. The off-topic prediction follows from the topical-coherence dependence reported in Old Habits Die Hard.
The second is whether the probe predicts $g_{t+1}$ better than the model's own answer entropy. We measure this by the area under the curve (AUROC) on held-out questions, and we also test transfer to 2WikiMultihopQA without retraining.
The third is end-to-end performance. We report EM and F1, retrieval recall, and average tokens on MuSiQue, 2WikiMultihopQA, and HotpotQA against the matched Stop-RAG baseline and the three ablations. We specifically check whether rollback removes Stop-RAG's HotpotQA shortfall.
A positive $\Delta$ with no end-to-end gain would still be a useful finding: persistence exists, but detecting it is too costly to pay off. A near-zero $\Delta$ would redirect the field toward per-hop filtering. Either outcome is informative.
References
- Old Habits Die Hard: How Conversational History Geometrically Traps LLMs
- Stop-RAG: Value-Based Retrieval Control for Iterative RAG
- Incremental Sequence Classification with Temporal Consistency
- Breaking Focus: Contextual Distraction Curse in Large Language Models
- Improving Zero-shot Reader by Reducing Distractions from Irrelevant Documents in Open-Domain Question Answering
- Beyond RAG vs. Long-Context: Learning Distraction-Aware Retrieval for Efficient Knowledge Grounding
- BRIEF: Bridging Retrieval and Inference for Multi-hop Reasoning via Compression
- Chain-of-Retrieval Augmented Generation
- ♫ MuSiQue: Multihop Questions via Single-hop Question Composition
- Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions
- HopRAG: Multi-Hop Reasoning for Logic-Aware Retrieval-Augmented Generation