A language model answering a question from retrieved documents is easily pulled off course by passages that look relevant but are not. The problem gets worse when the question is in one language and the documents are in another or in a mix: the model reasons less well, and it often slips into answering in English. The usual fixes train the model to resist distractors on preference data written by GPT-4o, and they have been tested only in English. The data that would teach the same skill in Swahili, Telugu or Arabic is the data nobody wants to pay to annotate.
We propose a way to make that training data for free. Take an unlabeled passage, have the model write a question about it in a target language, and record the model's answer when it sees only that passage. That clean answer becomes the "preferred" response. Then bury the same passage among retrieved distractors in several languages and record the model's answer again. When this noisy answer disagrees with the clean one, or comes out in the wrong language, it becomes the "dispreferred" response. Training on these pairs with direct preference optimization, a method that pushes the model toward the preferred of two responses and away from the other, teaches the model that adding irrelevant cross-lingual material should not change its answer. No human or teacher model labels anything. Which passage is relevant is known because we chose it.
Core Claim
The claim is that invariance to irrelevant context can be supervised by the model's own behavior on a context whose relevance is known by construction, and that this signal transfers across languages where labeled data does not exist. It is different from the closest work in a specific way. SSFO also builds label-free preference pairs, but its contrast is "answer with context" versus "answer from memory," which targets conflicts between the context and the model's prior knowledge. Its training data is English MS MARCO with single passages, and it does not test distractors or mixed-language contexts. Our contrast is "clean context" versus "same context plus distractors," which targets distraction itself. Ext2Gen shows that training against up to 25 distractor chunks is what makes a reader robust, but its preference data comes from GPT-4o and it is English-only. We keep its distractor-mixing recipe and remove the teacher.
The motivating evidence comes from XRAG. That benchmark found that reasoning over multiple documents in mixed languages, rather than fluent generation, is the main bottleneck in cross-lingual retrieval-augmented generation. It also found that Mistral-Large answered in the wrong language in more than 12% of German cases. XRAG proposed no mitigation.
Research Program
The construction pipeline starts from unlabeled Wikipedia text in the eleven TyDi QA languages and in English. Articles that appear in any evaluation set are held out.
For each source passage $p$ and target language $L$, the model writes a question in $L$ about $p$. Sometimes $p$ is itself in $L$. Sometimes $p$ is English and the question is in $L$; this reproduces XRAG's setting of a foreign query with English documents.
The clean answer is sampled several times from the pair (question, $p$). We keep an item only if most samples agree and a cheap language identifier confirms they are in $L$. This consistency filter is our answer to the label noise that CLTP shows plagues cross-lingual pseudo-labeling. CLTP's candidate-set method does not extend to free-form answers, so we use agreement across samples instead.
Distractors come from a multilingual retriever run on the generated question. They are drawn as same-language, English, and third-language passages, with $p$ excluded. A retrieved passage might secretly contain the answer, which would make it a false distractor. To catch these, we show the model each candidate distractor alone and drop it if the model reproduces the clean answer. This check is also label-free.
The noisy context mixes $p$ with $k$ distractors at random positions, with $k$ ranging up to 20, so the model also learns to resist "lost in the middle" effects. A pair is formed when the noisy answer differs from the clean answer or is in the wrong language. We expect a few thousand pairs per model to suffice, since SSFO reports that 400–500 self-generated examples capture over 85% of its gain.
The analysis side asks where the cross-lingual tax comes from. We measure the "distraction curve," meaning accuracy as a function of $k$, separately for same-language, English and third-language distractors. We then ask whether training flattens each curve. This answers an open question raised by CompAct's English-only results: is the damage from cross-lingual noise mostly volume (more tokens) or mostly language mismatch?
Baselines
The naive reference point is the base instruction-tuned model. We use Llama-3-8B and Mistral-7B, chosen to match SSFO, reading the full top-$k$ retrieved context with an instruction to answer in the query's language. Its distraction curve and wrong-language rate are the quantities we aim to beat.
The prompting baseline is Distraction-aware Answer Selection from Improving Zero-shot Reader by Reducing Distractions. That method adds an "unanswerable" escape option and reweights answers by how likely each document is to generate the query. It reports a 64% average gain in exact match on English open-domain QA. We run it unchanged on the same retrieved contexts.
The strongest trained competitor is an Ext2Gen-style model. It uses the same DPO training and the same distractor mixing on the same source passages, but its preferred and dispreferred answers are judged by GPT-4o in the target languages. On English data, Ext2Gen reports an average score of 0.557 for Ext2Gen-R2 versus 0.473 for the default Llama3.1-8b-instruct (Ext2Gen). No reported result was found on Lacuna for Ext2Gen, CompAct or SSFO on XRAG or on multilingual distractor-augmented TyDi QA. We will therefore produce those numbers ourselves, holding pair count, DPO hyperparameter search budget and base model equal.
We also run released SSFO trained on English, plus SSFO retrained on our multilingual passages. The second version isolates whether our clean-versus-noisy contrast matters, or whether the gain comes from SSFO's context-versus-memory contrast applied to multilingual data.
Four ablations isolate the key ingredients:
- Removing the false-distractor filter.
- Using only same-language distractors.
- Dropping the wrong-language criterion from pair selection.
- Replacing clean-context answers with gold TyDi answers on a small labeled subset. This gives an oracle that bounds the cost of self-supervision.
Beating the field means two things. First, matching or exceeding the GPT-4o-labeled model on XRAG accuracy and TyDi F1 at $k=10$ and $k=20$. Second, a measurably flatter distraction curve for cross-lingual distractors. Both must hold without losing more than a point on English NQ.
Why not just…?
Why not just translate the query to English and use English noise defenses? Translate-test is a serious competitor, and we run it with the same reader. It uses NLLB-class open translation into English, then the English Ext2Gen-style model, then translation back. The evidence against it is real but indirect. On XNLI, translate-test still trails English by 3–6 points (XNLI). TyDi QA shows that translation distorts native questions, which have much lower lexical overlap with their passages (0.26 in Arabic versus 1.29 for translated XQuAD). Translation also does nothing for mixed-language document sets. If translate-test matches us on TyDi but not on XRAG's mixed-retrieval setting, that tells us exactly where native training matters.
Why not just retrieve fewer documents? We report every method at $k=1$. CompAct shows that multi-hop recall needs deeper retrieval, and XRAG's hard cases need multiple documents. Still, PARC found $k=1$ near-optimal for low-resource classification. If $k=1$ wins on single-hop TyDi, we will say so, and the method's value would then rest on the multi-document XRAG setting.
Why not just filter passages with a multilingual reranker before the reader sees them? This is the most dangerous alternative. If a good enough reranker exists, distraction becomes a retrieval problem. Retrieval or Representation? shows that better multilingual preprocessing alone gives +8.9 Top-5 points, which hints that many "reader" failures start upstream. We therefore include a cross-encoder reranker plus top-3 baseline and an oracle-filter upper bound. Reader training is justified only if it adds accuracy on top of the reranker. If it does not, the honest conclusion is to invest in retrieval instead.
Why not just tell the model to say "unanswerable" or to answer in the query language? SKR found that asking the model directly about its own knowledge was the weakest strategy it tested. XRAG's wrong-language failures occurred despite explicit instructions. The DAS baseline tests this directly.
Proposed Experiments
The main evaluation has three parts:
- XRAG. We use its monolingual-retrieval and multilingual-retrieval settings in Arabic, Chinese, German and Spanish, scored with XRAG's accuracy protocol. Reference points from that paper are 85% for humans and 62% for GPT-4o on English, and some models drop more than 50% in Arabic. We also report the wrong-language rate.
- Distractor-augmented TyDi QA. We bury the gold passage among $k$ retrieved same-language, English and mixed passages. Scoring is minimal-answer F1, alongside the original mBERT baseline of 50.5 F1 and the human estimate of 70.1 for scale. Results are split by high- and low-resource languages.
- English regression checks. We test on NQ and NQ-Swap.
Two checks are decisive. The first is whether the self-supervised model closes most of the gap to the GPT-4o-labeled model and to the gold-label oracle on Swahili, Telugu and Bengali, where the teacher is weakest. The second is whether gains survive the reranker baseline at matched context length. A third diagnostic tests the mechanism. If training flattens the curve for English distractors under non-English queries much more than the curve for same-language distractors, then the cross-lingual tax is largely learned language-mismatch distraction, not missing knowledge. If it does not, the bottleneck lies in parametric knowledge, and selective retrieval in the style of SKR would be the better next step.
References
- SSFO: Self-Supervised Faithfulness Optimization for Retrieval-Augmented Generation
- Aligning Extraction and Generation for Robust Retrieval-Augmented Generation
- XRAG: Cross-lingual Retrieval-Augmented Generation
- Uncertainty-Aware Cross-Lingual Transfer with Pseudo Partial Labels
- CompAct: Compressing Retrieved Documents Actively for Question Answering
- Improving Zero-shot Reader by Reducing Distractions from Irrelevant Documents in Open-Domain Question Answering
- XNLI: Evaluating Cross-lingual Sentence Representations
- TyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languages
- Cross-Lingual Retrieval Augmented Prompt for Low-Resource Languages
- Retrieval or Representation? Reassessing Benchmark Gaps in Multilingual and Visually Rich RAG
- Self-Knowledge Guided Retrieval Augmentation for Large Language Models