Bridge-Entity Corruption: Synthetic Errors That Match How Multi-Hop Reasoners Actually Fail

When a language model answers a question that needs several lookups ("In what city was the director of Inception born?"), it usually fails in the middle. It latches onto the wrong intermediate entity, or binds the right entity to the wrong relation, and every later step builds on that slip. A checker that reads each step and flags these slips would help. Training one requires many examples of wrong steps, and the cheap way to get them is to corrupt correct reasoning on purpose, which is called synthetic error injection. The literature disagrees about whether this works. Models fine-tuned on injected errors fix 99% of them yet recognize only 8% of their own natural mistakes (Synthetic Error Injection Fails to Elicit Self-Correction In Language Models). Hallucination classifiers trained on synthetic corruptions do reach 95.3% ROC AUC on organic errors (Detecting Hallucination and Coverage Errors in Retrieval Augmented Generation for Controversial Topics).

We propose that the deciding factor is whether the injected errors look like the reasoner's real errors. In multi-hop retrieval, the real errors are mostly structural. The model picks the wrong "bridge" entity, usually one that appeared in a passage it retrieved, or it swaps who-did-what-to-whom. We build a step-level verifier trained on bridge-entity corruptions. These are made by parsing a correct intermediate step into its predicate and arguments, then replacing an argument with an entity of the same type drawn from the distractor passages the reasoner actually retrieved at that step. The parser runs only offline, to make training data. Its mistakes therefore become measurable label noise and never reach inference, which avoids the upstream-parser fragility that limits parse-based question decomposition.

Core Claim

Synthetic negatives that target the hop structure and are drawn from the reasoner's own retrieved context will produce a step verifier whose accuracy on natural errors approaches that of a verifier trained on genuinely on-policy labeled errors. Here "on-policy" means mistakes the reasoner itself produced, as opposed to errors injected into someone else's correct text. Token-level swaps and free-form LLM-generated errors will show the large synthetic-to-natural drop reported by Synthetic Error Injection Fails to Elicit Self-Correction In Language Models. We make one quantity central. For a verifier trained on generator $g$, let $A_{\text{syn}}(g)$ be its step-level AUROC on held-out synthetic errors from $g$ and $A_{\text{nat}}(g)$ its AUROC on the reasoner's natural errors. The transfer ratio is

$$ \rho(g) = \frac{A_{\text{nat}}(g) - 0.5}{A_{\text{syn}}(g) - 0.5}, $$

the fraction of above-chance skill on synthetic errors that survives on natural ones. Our prediction is that $\rho$ tracks how closely $g$'s error-type distribution matches the natural one, and that bridge-entity corruption gives the highest $\rho$ among the label-free generators.

The proposal differs from three existing lines of work. It differs from the toy-domain study of Synthetic Error Injection Fails to Elicit Self-Correction In Language Models in two ways: it trains a separate detector rather than teaching the generator to self-correct, and its errors are relational rather than single-digit or single-cell swaps. It differs from Learning from Synthetic Data Improves Multi-hop Reasoning, which synthesizes only valid multi-hop chains and rewards final outcomes, by synthesizing invalid intermediate steps. It differs from Adversarial Training for Process Reward Models, whose free-form adversarial generator could not produce precondition violations in math. In multi-hop text the precondition is concrete: the entity from hop $k$ must fill the argument slot that hop $k+1$ needs. We construct violations of that condition directly instead of hoping a generator discovers them.

Research Program

The first stage builds a measurable notion of a natural error. MuSiQue builds each question by composing single-hop questions, so every hop has a gold sub-question and a gold intermediate answer (MuSiQue: Multi-hop Questions via Single-hop Question Composition). It was also filtered against single-hop shortcuts that let models skip intermediate hops on HotpotQA and 2WikiMultihopQA. We run an interleaved retrieve-and-reason system in the style of IRCoT (Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions), which alternates one reasoning sentence with one retrieval issued from that sentence. We run it with two open models of different sizes and align each generated step to a gold hop. A step is naturally wrong if its bridge entity disagrees with the gold intermediate answer or its claim is unsupported by the gold paragraph. A sample of these steps, about 1,000, is hand-categorized into wrong bridge entity, argument swap, wrong relation, skipped hop, and fabricated fact. This yields the natural error-type distribution and a gold-labeled natural test set.

The second stage builds four families of error generators, each applied to correct steps. The first is token-level operator and entity swaps in the style of REFINER, a method that perturbs math reasoning chains to train a critic (REFINER: Reasoning Feedback on Intermediate Representations). The second is free-form LLM rewriting, as used to inject hallucinated attributes in Hal-Eval: A Universal and Fine-grained Hallucination Evaluation Framework for Large Vision Language Models. The third is mask-and-fill, in which key spans are hidden and a generator fills them plausibly but wrongly (Masked Summarization to Generate Factually Inconsistent Summaries for Improved Factual Consistency Checking). The fourth is our bridge-entity corruption. An off-the-shelf dependency and semantic-role parser identifies the predicate and its arguments in the step, which tells us which span is the bridge entity and which spans fill which roles. We then apply one of three edits. We can replace the bridge with a same-type entity taken from that step's retrieved but non-gold passages. We can swap two arguments. Or we can drop the hop by substituting the final-hop entity early.

The third stage trains one verifier architecture per generator family. The verifier is a mid-sized encoder or small decoder that reads the question, prior steps, current step, and retrieved passages and outputs a probability that the step is wrong. All variants share the same base model, number of training examples, and tuning budget.

The fourth stage studies the parser as a source of noise. Following the observation in Revisiting the Case for Explicit Syntactic Information in Language Models that syntax-augmented systems are capped by parser error, and the evidence in Interpretable AMR-Based Question Decomposition for Multi-hop Question Answering that a bad parse yields broken sub-questions, we degrade the parser in controlled steps. We swap in weaker parsers and randomly reattach a fraction of dependency edges, then measure how many generated "negatives" are actually still correct and how verifier quality on natural errors declines. Because the parser is used only offline, we expect graceful degradation, unlike the hard ceiling that inference-time parse dependence imposes. This is a falsifiable prediction.

Baselines

The naive reference point is the unverified IRCoT reasoner, which accepts every step. We also include a trivial verifier that scores each step by its lexical overlap with retrieved passages, the ROUGE-style signal that reached about 84–85% AUC on single-hop retrieval summaries in Detecting Hallucination and Coverage Errors in Retrieval Augmented Generation for Controversial Topics. It is cheap and has to be beaten.

Two baselines are the strongest competitors, and both exploit on-policy data. The first is an on-policy verifier trained directly on MuSiQue's naturally wrong steps, labeled with gold intermediate answers. It sees exactly the target distribution, so it serves as the ceiling. The claim succeeds if bridge-entity corruption reaches most of its natural-error AUROC with no intermediate labels, and decisively if this also holds when both verifiers are moved to HotpotQA and 2WikiMultihopQA. The second is an outcome-derived implicit step reward in the style of DPRM (DPRM: A Dual Implicit Process Reward Model in Multi-Hop Question Answering). DPRM derives step rewards from final-answer correctness using a preference-style likelihood ratio over chains, and it reports 0.5988 F1 on Complex WebQuestions. That result is over a knowledge graph, and we found no reported result on Lacuna for DPRM on text-only multi-hop corpora such as MuSiQue. We adapt only its text-chain component, train it on the same reasoner's samples with only final-answer labels, and match the number of training chains and compute. This is the realistic label-free competitor. Beating it means higher step-level AUROC on gold natural errors and a larger end-task F1 gain when used for step rejection.

The prior-method baselines are the REFINER-style, free-form, and mask-and-fill generators, all trained identically. For end-task context, the best number the file reports on MuSiQue-Ans is 47.3 F1 for the Select+Answer model, against 78.0 human F1 (MuSiQue: Multi-hop Questions via Single-hop Question Composition). For IRCoT the file reports gains of up to 15 F1 and 11–21 recall points but no absolute MuSiQue F1, so we will report our own reproduction.

Two ablations isolate the key ingredient. The first draws replacement entities at random from the corpus, matched for type, instead of from the step's retrieved distractors. This tests whether "near-on-policy" sourcing matters or whether any structural edit suffices. The second keeps distractor sourcing but replaces the parser with an LLM instructed to "swap the entity that links to the next hop." This tests whether explicit structure is needed at all.

Why not just…?

Why not just train on the reasoner's own labeled mistakes? On MuSiQue this works and is our ceiling. But it needs gold intermediate answers, which almost no real retrieval corpus has; HotpotQA and 2WikiMultihopQA provide supporting paragraphs, not intermediate answers by construction. The cross-dataset transfer experiment shows whether a MuSiQue-trained on-policy verifier carries over. If it transfers as well as our method, the honest conclusion is that a small labeled on-policy set beats synthetic generation, and we will report that.

Why not just derive step labels from final-answer correctness? This is the most serious threat, because it is label-free and fully on-policy. Its weakness is credit assignment. A wrong final answer does not say which step broke, and multi-hop chains can reach right answers through shortcut paths, as MuSiQue: Multi-hop Questions via Single-hop Question Composition documents. Step-level AUROC against gold intermediate labels measures this directly. If the outcome-derived reward matches ours, synthetic structure adds nothing and the proposal fails.

Why not just ask a strong LLM to judge each step against its retrieved passages? Zero-shot judging needs no training, but unguided self-checking shows the "parrot" behavior in Synthetic Error Injection Fails to Elicit Self-Correction In Language Models, and REFINER: Reasoning Feedback on Intermediate Representations found that GPT-3.5 improved only when a trained critic supplied structured feedback. A wrong bridge entity that does appear in a retrieved passage is exactly the case where a passage-consistency judge should be fooled. We will run a prompted judge of larger size than our verifier on the same natural test set and report it by error type. This is where we expect the trained verifier to win.

Why not just skip the parser and let an LLM write the structural errors? This is the second ablation. If LLM-instructed swaps match parser-guided ones, the parser is unnecessary and the parser-noise study becomes moot. That would still be a useful finding, because it would show the gain comes from distractor sourcing rather than from syntax.

Proposed Experiments

The primary measurement is step-level AUROC on gold-labeled natural errors from MuSiQue, reported for each generator family and broken down by error type, together with the transfer ratio $\rho$. The decisive check has two parts. Bridge-entity corruption should have the highest $\rho$ among the label-free generators. It should also beat the outcome-derived implicit reward on natural-error AUROC. If the random-entity ablation matches it, the "own retrieved context" hypothesis is wrong.

A second check correlates each generator's error-type distribution, measured by total-variation distance from the hand-categorized natural distribution, with its $\rho$. A strong negative correlation would explain the conflict between the toy-domain failure and the retrieval-detection success as a matter of distribution match, not of synthetic data being good or bad in general.

Downstream, each verifier gates IRCoT. A flagged step triggers re-retrieval with the flagged entity excluded, or resampling of the step, under a fixed budget of extra model calls. We report answer F1 and exact match on MuSiQue-Ans, HotpotQA, and 2WikiMultihopQA. We train only on MuSiQue so that the other two datasets test transfer.

The parser-degradation curve plots generated-negative precision and natural-error AUROC against parser accuracy. A flat or gently sloping curve supports moving the parser offline. A steep drop would mean upstream parser dependency survives even as a data generator.