Language Is Not Relevance: Synthetic Multilingual Worlds for Noise-Robust Cross-Lingual Multi-Hop QA

Suppose a small language model answers a question in Hindi from a pile of retrieved documents. Some documents are in Hindi, some in English, some in Chinese, and most are irrelevant. To answer, the model has to chain two or three facts together, a skill called multi-hop reasoning. We conjecture that in this setting compact models fall back on a cheap heuristic: a passage written in the question's language is probably the relevant one. If so, the model is misled twice. It is drawn toward same-language distractors, and it overlooks the bridging fact when that fact appears only in another language. Nobody has measured this, because every training method that makes small models robust to irrelevant context is English-only. The one cross-lingual multi-hop system in the literature relies on an external retriever and a frozen model rather than training the reader.

We propose to generate training data from rule-built fictional worlds, the kind of synthetic data that already teaches small models English multi-hop reasoning, and render each world in many languages. Because the generator controls everything, we can assign each document's language independently of whether it is relevant. That breaks the link between "same language" and "useful" that we suspect real data rewards. We then train with reinforcement learning on two signals that can be checked by a program: whether the final answer is correct, and which passage identifiers the model cites as evidence. The model may think in any language it likes, but it must point to the right passages.

Core Claim

Our first claim is that language match is a measurable relevance shortcut in small-model readers working over multilingual retrieved context. Our second claim is that training on synthetic worlds where document language is decorrelated from relevance removes the shortcut and transfers to real cross-lingual multi-hop benchmarks. It should do so better than English-only synthetic training, better than translating everything into English, and better than distilling a teacher's reasoning traces. We quantify the shortcut with a language-match sensitivity

$$S_{\text{lang}} = \text{Acc}(\text{distractors in other languages}) - \text{Acc}(\text{distractors in query language}),$$

where Acc is answer accuracy. The two conditions use the same distractor content, translated, so the only thing that changes is which language the distractors are written in. A companion quantity, the foreign-evidence penalty $P_{\text{for}}$, is the accuracy drop when the bridging fact appears in a non-query language instead of the query language. A shortcut-free reader has both quantities near zero.

What Prior Work Leaves Open

Two papers supply the method. Learning from Synthetic Data Improves Multi-hop Reasoning trains Qwen3 and Phi-4 models of 0.6B–7B parameters with Group Relative Policy Optimization (GRPO), a reinforcement learning method that compares groups of sampled answers to the same question. The training data are rule-generated fictional worlds such as PhantomWiki, and the reward checks only the final answer. It reports 56% to 131% relative F1 gains for Qwen3-0.6B on HotpotQA, 2WikiMultihopQA, MuSiQue and CofCA. Its data and evaluation are English-only and contain no distractors, which is exactly the space we fill. Beyond English-Centric Training shows that GRPO beats supervised fine-tuning by 17.5–25.2 points on the multilingual math benchmark MGSM. It also finds that German-trained RL reaches 71.5% averaged over 10 languages, against 62.7% for English-trained RL. That result motivates rendering synthetic worlds natively in non-English languages instead of assuming English is the best training language. Its tasks are clean, single-context math problems, so it tells us nothing about retrieval noise.

Two papers motivate the problem and set the bar on noise robustness. Robust Knowledge Editing via Explicit Reasoning Chains for Distractor-Resilient Multi-Hop QA (Reason-KE) fine-tunes Qwen2.5-7B on DeepSeek-R1 traces that label each fact "Apply" or "Ignore". It reaches 90.2% on MQuAKE-CF and loses only 6.3% with four distractor facts. However, it is English-only, it depends on a teacher model, and it uses at most four short distractors. Aligning Extraction and Generation for Robust Retrieval-Augmented Generation (Ext2Gen) uses preference training so that Llama3.1-8b-instruct extracts evidence sentences before answering amid up to 25 distractor chunks. This raises its score from 0.473 to 0.557, but the evaluation is English-only and requires copying sentences verbatim.

That copying requirement creates a tension, laid out in the file, with the RL finding that models gain from switching languages mid-reasoning. A reader that must copy grounding sentences cannot freely reason in English about a Bengali passage. Our evidence-by-pointer reward is designed to resolve this tension. Grounding is checked through passage identifiers, which mean the same thing in every language, so the language of the reasoning is left unconstrained.

Two papers constrain the design. Cross-Lingual Multi-Hop Knowledge Editing introduces CROLIN-MQUAKE across eight languages and the CLEVER-CKE pipeline, which uses a frozen model, an external multilingual retriever and a fixed cosine-similarity filter. It reports up to 30% gains over MeLLo and PokeMQA and a clear gap between Latin and non-Latin scripts. It never trains the reader, which leaves open whether the reader itself can be made robust. Bridging the Language Gap finds that cross-lingual multi-hop failures come from misrouted attention heads rather than missing knowledge. It also shows that full fine-tuning and DPO can collapse MMLU by more than 60 points, whereas tuning about 40 heads does not. That result obliges us to monitor general capability and to compare against restricted-parameter training.

Research Program

The first component is a multilingual synthetic generator. We start from PhantomWiki-style worlds: fictional people and relations, articles generated from templates, and multi-hop questions with exactly computable answers. The template set is small, a few dozen sentence and question forms. Fluent speakers, with model assistance, can therefore translate it natively into the eight CROLIN-MQUAKE languages: English, German, Spanish, Chinese, Russian, Hindi, Bengali and Swahili. Fictional names are transliterated per script, so tokenizer fragmentation in non-Latin scripts is part of the training data.

Each training instance is a question in language $q$ plus a context of up to 30 passages. The context contains the gold passages for every hop. It also contains distractors of three kinds: random passages, passages about the same entities with the wrong relation, and passages that complete a plausible but wrong chain. The generator assigns each passage's language under one of three placement regimes. In the natural regime, gold passages are mostly in $q$, which mimics monolingual corpora. In the decorrelated regime, language is drawn independently of relevance. In the adversarial regime, distractors are preferentially in $q$ and gold passages preferentially are not. We train on a mix of the decorrelated and adversarial regimes. The natural regime is kept as a control.

The second component is the reward. The model's output ends with an answer and a list of cited passage identifiers. The reward is exact-match correctness of the answer plus a bonus for the F1 overlap between cited and gold identifiers. The bonus is gated on a correct answer, so that citing evidence correctly cannot be rewarded while the answer is wrong. Reasoning may mix languages freely. The final answer must be in $q$, and a language-identification check enforces this, because the file notes that unconstrained mixing can degrade user-facing output. Both parts of the reward are computed by a program, and no teacher traces are used.

The third component is analysis. We measure $S_{\text{lang}}$ and $P_{\text{for}}$ on base models before any training. We also compute attention mass on same-language versus other-language passages. Following the head-level analysis in Bridging the Language Gap, we ask whether training changes which heads route attention toward query-language passages.

Students are Qwen3-0.6B, Qwen3-1.7B, Qwen3-4B and Qwen2.5-7B. The last allows a direct comparison with Reason-KE.

Baselines

The do-nothing reference is the untrained instruction-tuned student, given the same noisy multilingual context and a standard chain-of-thought prompt. A second trivial reference adds the negation instruction from Distraction-aware Answer Selection, which tells the model to return "unanswerable" if unsure. That method is training-free and reported a 64% average Exact Match gain over naive zero-shot readers on single-hop open-domain QA. Any trained method that cannot beat this prompt is not worth its compute.

The closest prior training recipe is English-only synthetic GRPO as in Learning from Synthetic Data Improves Multi-hop Reasoning. We run it in two forms. The first is its original form, with clean English PhantomWiki. The second adds our English distractors, which separates the value of noise from the value of multilinguality. Its reported result is 56–131% relative F1 for Qwen3-0.6B on English benchmarks, and no reported result was found on Lacuna for it on any cross-lingual benchmark.

The strongest distillation competitor is Reason-KE-style supervised fine-tuning on teacher traces. We generate four-stage Apply/Ignore traces with DeepSeek-R1 over our own multilingual synthetic worlds, so the data are identical to ours and only the learning signal differs. Its reported reference point is 90.2% on English MQuAKE-CF with Qwen2.5-7B and a 6.3% drop under four distractors (Reason-KE).

The strongest pipeline competitor is CLEVER-CKE (Cross-Lingual Multi-Hop Knowledge Editing), run on CROLIN-MQUAKE with the same distractor pools. The file reports only a relative figure of up to 30% over MeLLo and PokeMQA. No absolute CROLIN-MQUAKE accuracy for CLEVER-CKE was found on Lacuna, so we will reproduce it with the released configuration.

Fairness rules apply to all trained baselines. Every one starts from the same checkpoint, sees the same number of training questions drawn from the same worlds, and receives a matched GPU-hour budget. Supervised fine-tuning is allowed extra epochs to equalize compute with RL. Every method gets the same hyperparameter sweep of eight learning-rate and KL settings, chosen on a held-out synthetic split, never on the test benchmarks.

Two ablations isolate our ingredients. The first is placement: an identical run that differs only in using the natural placement regime. This isolates decorrelation and is the most important comparison in the proposal. The second is reward: the same run without the evidence-pointer term. Beating a baseline means a gain on the real cross-lingual benchmarks that exceeds seed variance across three seeds. It also means a $S_{\text{lang}}$ reduced to within two points of zero, with English MuSiQue F1 and MMLU within one point of the base model.

Why not just…?

Why not just translate everything into English and run an English-robust reader? This translate-test pipeline is the real threat, because English-only synthetic RL plus machine translation of the query and all passages might erase the language shortcut entirely. The file's evidence against it is indirect. Translation fluency is known not to equal reasoning capacity, and per-hop translation compounds error in multi-document settings. Neither claim has been measured on noisy multi-hop retrieval. We will run translate-test with a strong translation model and give it its own latency and compute accounting. If it matches our method on non-Latin-script languages, we will report that translation is the practical answer and our contribution narrows to cost.

Why not just train on English synthetic data and rely on RL's known cross-lingual transfer? Beyond English-Centric Training shows that RL reasoning transfers across languages on clean math prompts. English PhantomWiki-GRPO might therefore already be robust. However, a model never trained on mixed-language contexts has had no pressure to stop preferring same-language passages. The English-with-distractors baseline tests this directly. If its $S_{\text{lang}}$ is already near zero, our central hypothesis is false.

Why not just distill a strong teacher's filtering traces? Reason-KE shows this works in English at small distractor counts. The same paper leaves open whether it scales to dozens of passages, and supervised fine-tuning on traces shows negative out-of-distribution transfer in the cross-lingual RL study. Our matched-data supervised fine-tuning baseline settles the question on identical worlds.

Why not just put a multilingual reranker in front of the reader? Static filters judge each passage against the original question. They therefore tend to discard bridging facts whose relevance only becomes apparent after the first hop, and Ext2Gen already outperforms compression filters such as Recomp, CompAct and EXIT in English. A cross-lingual reranker may itself carry the language-match bias. We include a multilingual reranker that keeps the top 5 passages in front of the untrained reader, and we measure the reranker's own $S_{\text{lang}}$.

Proposed Experiments and Decisive Checks

The first experiment is purely diagnostic and gates everything else. We measure $S_{\text{lang}}$ and $P_{\text{for}}$ for base Qwen3 and Qwen2.5 students on two test sets. The first is CROLIN-MQUAKE, with each query's needed facts placed in context alongside 4, 10 and 25 distractor facts in controlled languages. The second is a mixed-language MuSiQue variant, built by machine-translating MuSiQue paragraphs into the eight languages and assigning languages by the three placement regimes. MuSiQue is chosen because its composition filters block single-hop shortcuts, and we treat translation artifacts as noise common to all methods. If both quantities are near zero at every scale, the shortcut does not exist. The project then reduces to multilingual noisy synthetic RL, and we report that honestly.

The main experiment trains each student under our recipe and every baseline. We report answer F1 and evidence-ID F1 on the mixed-language MuSiQue variant, accuracy on CROLIN-MQUAKE-CF and CROLIN-MQUAKE-T broken down by script, and $S_{\text{lang}}$ and $P_{\text{for}}$. Clean English MuSiQue, HotpotQA and 2WikiMultihopQA check that English ability is retained. XHOPREASON checks that parametric cross-lingual reasoning does not degrade, and MMLU checks general capability.

The decisive comparison is the placement ablation evaluated on real data. The hypothesis survives if decorrelated training lowers $S_{\text{lang}}$ on real CROLIN-MQUAKE and MuSiQue-variant contexts, not just on synthetic ones, and beats natural-placement training on non-query-language evidence. A secondary test trains separate models on worlds rendered only in English, only in German, and only in Hindi. It asks whether the non-English training advantage seen for MGSM carries over to synthetic multi-hop worlds. A final stress test scales distractors to 25 full passages, beyond Reason-KE's four facts. It asks whether the evidence-pointer reward keeps accuracy flat as noise grows.