Multi-hop question answering systems answer questions like "Where was the director of Inception born?" by chaining lookups: they find the director, then use that name to search for the birthplace. Modern systems of this kind interleave a language model's step-by-step reasoning with repeated searches over a document collection, a setup called iterative retrieval-augmented generation (RAG). Each search depends on what the previous step concluded. Separately, research on text robustness has found that defenses built for one kind of edit fail against other kinds. A defense against synonym swaps does little against typos, and neither handles rewritten phrases. This note argues that these two facts interact in a specific and underexplored way. The granularity of an edit, meaning whether it changes characters, words, or phrases, determines which stage of a multi-hop chain it breaks. The most dangerous edits are not the loud ones. They are small phrase-level edits that make the system confidently pick the wrong intermediate entity, the "bridge". Every later retrieval is then conditioned on that wrong bridge.
The central mechanism is what we call coherent divergence. Once hop one returns a wrong bridge, all evidence retrieved at hop two agrees with that wrong bridge. Defenses that look for consensus among retrieved passages therefore see a clean, unanimous, and wrong context. So does self-consistency voting over sampled chains, and so do relevance filters. We propose to measure this failure across perturbation granularities and hop positions. We also propose to test one targeted remedy, provenance-disjoint bridge corroboration: a bridge entity is accepted only if a second passage supports it, and that passage must have been retrieved by a query that never saw the text which proposed the bridge.
Core Claim
We make two claims. The first is diagnostic. In iterative RAG, character-level edits to a bridge entity mostly cause retrieval misses: the search finds nothing useful, and the system often notices and retries or abstains. Phrase-level edits to the relation words around an intact entity instead cause confident wrong bridges, and the system never notices. The second failure is mostly invisible to within-hop defenses. The second claim is constructive. Requiring independent corroboration for each bridge recovers much of the lost accuracy at modest cost. It does better than simply sampling more reasoning chains, because sampled chains share the same corrupted retrieval.
Where This Sits Relative to Prior Work
The closest prior work is Reasoning Chain Based Adversarial Attack for Multi-hop Question Answering. It keeps entities fixed, swaps the relational phrase (for example, "born in" becomes "named in"), and shows that pre-LLM reading models on fixed HotpotQA distractor sets collapse: DFGN's exact match falls from 55.9% to 25.6%. It motivates our phrase-level threat. However, it studies a single granularity, uses no retriever in the loop, and offers only adversarial retraining as a defense. We move this attack into iterative retrieval, compare it against other granularities at matched edit budgets, and trace its effect hop by hop.
Three single-hop studies establish that RAG is fragile, and each leaves the multi-hop cascade open.
- Toward Robust RALMs injects whole misleading passages (its GenADV attack) into single-turn QA and shows that prompting does not fix the damage.
- Poisoning Retrieval Corpora by Injecting Adversarial Passages optimizes token-level passages that hijack dense retrievers. Its code, princeton-nlp/corpus-poisoning, will supply our passage-level attack.
- RAGSentinel filters retrieved passages by their distance from a geometric-median consensus. It certifies recovery only if fewer than half of the retrieved passages are poisoned.
That honest-majority assumption is exactly what coherent divergence breaks at hop two. RAGSentinel therefore constrains our design. It is the strongest defense of its type, and our first experiment asks directly whether its guarantee fails in the multi-hop setting.
On the robustness side, Revisiting Character-level Adversarial Attacks for Language Models introduces Charmer, a greedy search over character edits within a budget on Levenshtein distance (the number of single-character insertions, deletions, or substitutions). It reaches over 95% attack success on classifiers and LLMs, and shows that typo-correctors are brittle. ValCAT produces fluent, variable-length span rewrites. These papers supply our character-level and phrase-level attack generators. Both were built for classification, so we adapt their objectives to "change the bridge the system extracts" rather than "change a label."
On the systems side, we reuse existing multi-hop pipelines as the systems under test:
- IRCoT feeds each reasoning sentence back to the retriever as a new query.
- HANRAG filters out irrelevant passages with a relevance discriminator. It is designed against benign noise.
- MEGRAG escalates from compact triples to sentences to passages once evidence looks sufficient. Its authors note that it cannot backtrack.
We expect each to amplify phrase-level bridge edits. HGRAG uses "cross-granularity" in the representational sense (entities versus passages), not in the perturbation sense. It is a natural secondary target, since character edits to entity names could break its entity-passage links.
Research Program
The first component is a perturbation suite for multi-hop corpora. It is organized by granularity and by hop position. For each question in MuSiQue, 2WikiMultihopQA, and HotpotQA, the gold decomposition identifies the supporting passage for each hop. The attacker edits that passage, or inserts a passage next to it, at one of four granularities:
- Character: Charmer-style edits confined to the bridge-entity mention.
- Word: synonym substitutions on relation words.
- Phrase: ValCAT-style and Ding-style relational span rewrites that keep the entity but redirect the relation toward a decoy entity that exists in the corpus.
- Passage: corpus-poisoning insertion.
Budgets are matched across granularities by rejecting any perturbation whose semantic similarity to the original falls below a shared threshold. A human-judged sample checks that the edits stay plausible. Every edit is applied at hop 1, at hop 2, or at the final hop. This lets us separate granularity effects from position effects.
The second component is cascade instrumentation. For each run we log the bridge the system extracts at every hop and compare it to the gold bridge. This classifies each failure as one of three kinds: a miss (no bridge found, or the system abstains), a recovered miss (a later retry finds the right bridge), or coherent divergence (a wrong bridge is followed by evidence that agrees with it). We also record the fraction of hop-two passages that support the wrong bridge. This is a direct test of whether RAGSentinel's honest-majority condition holds.
The third component is the defense. At each hop, the system keeps its top two candidate bridges. For each candidate it issues a corroboration query built from the original question and the candidate entity alone. The query excludes the passage that proposed the candidate and the model's generated reasoning. A candidate is accepted only if a passage in the corroboration results entails the claimed relation, according to a natural language inference check. That passage must also come from a different source document than the proposing passage. Entity mentions are matched to a corpus-derived entity index using character-level fuzzy matching. This undoes typo-style damage without a general spell-checker, which would "correct" rare names into common words. If neither candidate is corroborated, the system backs off to the next candidate rather than committing. This gives MEGRAG-style pipelines the backtracking their authors say they lack.
Baselines
The naive reference point is IRCoT with no defense, run on clean and perturbed corpora with the same retriever (BM25 and Contriever), the same reader models (Llama-3.1-8B and Qwen-2.5-7B, matching RAGSentinel's model families), and the same step limit. This anchors every other number. The file reports that IRCoT improves QA F1 by up to 15 points over one-step retrieval. It does not report an attacked multi-hop score; no reported result found on Lacuna for IRCoT under adversarial corpus edits.
The strongest existing defense is RAGSentinel applied after every hop. It is training-free, it is the only defense in the file with a certified guarantee, and on single-hop NQ with Llama-3.1 its authors report attack success reduced from 0.38 (standard RAG) to 0.03. We run it with the authors' surrogate encoder and default radius, tuned on the same held-out development split used for our method.
The strongest multi-hop system that claims noise resistance is HANRAG with its relevance discriminator. It reports roughly a 7% exact-match gain and over 16% accuracy gain over Adaptive-RAG on clean MuSiQue. We use it both as a system under attack and as a filtering defense. MEGRAG, which reports +8.34 F1 over its strongest baseline on clean MuSiQue, serves as the most capable clean-accuracy system under attack. We also run self-consistency: sample several IRCoT chains at temperature and take a majority vote, with the number of samples chosen so its LLM calls and retrieval calls match our defense's measured budget.
Two ablations isolate the key ingredient. The first removes provenance-disjointness: corroboration queries may reuse the proposing passage and the generated reasoning. The second removes fuzzy entity matching.
Beating a baseline means higher attacked exact match and F1 at matched cost, with clean MuSiQue F1 no more than about 1.5 points below undefended IRCoT. Beating RAGSentinel specifically also means a lower coherent-divergence rate under phrase-level hop-1 edits.
Why not just…?
Why not just run a spell-checker over the corpus and queries? This would address only the character-level condition. Charmer's authors show that typo-correctors depend on narrow assumptions, and fail once attackers relax them. In multi-hop data the bridge entities are often rare names that a dictionary-based corrector will alter. We include a spell-check preprocessing baseline and report its clean accuracy as well. We expect it to lower clean MuSiQue F1 on rare-entity hops and to do nothing against phrase edits, which contain no misspellings.
Why not just apply RAGSentinel or HANRAG's discriminator at every hop? This is the most serious competitor, and experiment one tests it directly. Both judge each hop's retrieved set in isolation. If hop one's wrong bridge makes hop two's evidence unanimous, a consensus filter has nothing to reject. If our logging shows that per-hop RAGSentinel catches phrase-level hop-1 edits about as often as passage-level poisoning, the coherent-divergence claim is wrong. In that case we would report the benchmark and drop the defense.
Why not just sample more reasoning chains and vote? Sampled chains draw on the same corrupted passages, so they may agree on the same wrong bridge. That said, sampling with diverse queries could partly imitate corroboration, so this is a real threat. The no-provenance-disjointness ablation, compared against self-consistency at matched calls, settles it. If diverse sampling matches our method, the contribution becomes diagnostic and we will say so.
Why not just adversarially fine-tune the reader on mixed-granularity edits? Ding et al. show that retraining helps on the attack it was trained on: exact match recovers from 29.9% to 41.9%. Charmer's analysis warns that hardening a model against one granularity can raise its vulnerability to another. We fine-tune on phrase-level data only and test on character-level edits, and the reverse. This checks whether one-sided hardening transfers to the pipeline setting.
Proposed Experiments
The first experiment is the full grid: four granularities × three hop positions × three datasets × two readers, with exact match, F1, attack success rate (whether the answer moves to the decoy or away from gold), and the three failure categories. The central prediction is that phrase-level hop-1 edits show the highest coherent-divergence rate and the smallest gap between clean and attacked model confidence. Character edits should instead produce mostly misses.
The second experiment compares MuSiQue with HotpotQA. MuSiQue filters out questions that can be answered through single-hop shortcuts: a single-paragraph model gets 32.0 F1 there, versus 64.8 on HotpotQA. The question is whether genuinely connected reasoning makes cascades worse, or whether shortcut use on HotpotQA hides them.
The third experiment runs all defenses on the grid at matched cost. The decisive check is that corroboration beats per-hop RAGSentinel and cost-matched self-consistency on phrase-level hop-1 edits, and holds up at the passage level, while staying within the clean-accuracy tolerance. A secondary check measures cost: the fraction of extra LLM calls on clean HotpotQA. Per MEGRAG, nearly 90% of HotpotQA questions resolve in one hop, so corroboration should rarely trigger backoff there.
References
- Reasoning Chain Based Adversarial Attack for Multi-hop Question Answering
- Toward Robust RALMs: Revealing the Impact of Imperfect Retrieval on Retrieval-Augmented Language Models
- Poisoning Retrieval Corpora by Injecting Adversarial Passages
- RAGSentinel: Certifiable Geometric Consensus for Robust Retrieval-Augmented Generation
- Revisiting Character-level Adversarial Attacks for Language Models
- ValCAT: Variable-Length Contextualized Adversarial Transformations Using Encoder-Decoder Language Model
- Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions
- HANRAG: Heuristic Accurate Noise-resistant Retrieval-Augmented Generation for Multi-hop Question Answering
- MEGRAG: Multi-Granular Evidence Graphs for Answer-Aware Multi-Hop RAG
- Cross-Granularity Hypergraph Retrieval-Augmented Generation for Multi-hop Question Answering
- ♫ MuSiQue: Multihop Questions via Single-hop Question Composition