Small language models can learn to chain facts together ("who is the mother of the person who founded X?") from entirely fictional, rule-generated documents. If they are then trained with reinforcement learning that rewards only correct final answers, that skill carries over to real English multi-hop question answering. We do not yet know whether the skill belongs to reasoning or to English. This proposal tests that directly. It builds a version of the fictional sandbox in which the surface language is a controlled variable: the same fictional world can be rendered in English, in English with its word order deliberately rearranged, or in a few real languages written from native templates. It then measures how much of the reasoning gain survives in languages the model was never trained on.
The question matters for two reasons. First, the result says whether "knowledge composition" learned in a sandbox is a language-independent capability or an English habit. Second, it decides a practical question for low-resource languages. If the skill transfers, a fully synthetic, cheaply renderable curriculum could stand in for multi-hop training data that does not exist in most languages. If it does not, we need to know which part of the surface form it is bound to: word order, script, or morphology.
Core Claim
Learning from Synthetic Data Improves Multi-hop Reasoning trains Qwen3 and Phi-4 models on PhantomWiki, a generator of fictional Wikipedia-style pages and grammar-generated questions. The training uses outcome-reward reinforcement learning (GRPO, a policy-gradient method that compares several sampled answers per question). The paper reports relative F1 gains of 56% to 131% on HotpotQA, 2WikiMultihopQA, MuSiQue and CofCA, along with generalization from 2–3 hops to 9 hops. It is evaluated only in English.
Our claim has two parts. First, the zero-shot cross-lingual transfer of this gain is uneven. It should be largest for target languages whose word order matches the English templates and smallest for verb-final languages such as Hindi, Korean, Bengali and Telugu. This would mirror the source-syntax overfitting that Learning Disentangled Semantic Representations for Zero-Shot Cross-Lingual Transfer in Multilingual Machine Reading Comprehension documented for span-extraction models, where mBERT's predicted answers violated target-language syntactic constituent boundaries 23.15% of the time on BiPaR. Second, most of that unevenness can be removed without writing target-language data. The fix is to render the same fictional worlds under rearranged English word orders, so that the reward cannot be earned by relying on English word order. If this holds, the synthetic-to-natural gap and the cross-lingual gap share a cause: the templates' fixed surface order. That cause is cheap to vary inside a sandbox and expensive to vary in natural data.
The idea is distinct from earlier synthetic cross-lingual QA work. Synthetic Data Augmentation for Zero-Shot Cross-Lingual Question Answering, PAXQA and CLASS all generate natural-looking single-hop questions from real Wikipedia or bitext, using supervised training. We use fictional, rule-generated multi-hop worlds with verifiable answers, train with reinforcement learning, and treat linguistic surface form as an experimental knob rather than something to imitate.
Research Program
The first component is a multilingual extension of PhantomWiki that keeps the generator's world graph fixed (people, relations, facts) and swaps only the renderer. We will build three kinds of renderer. The first is the original English templates. The second is a set of "typologically rearranged English" renderers. These use the same English words, reordered to verb-final and verb-initial order, with postpositions instead of prepositions and with suffix-style possessive marking. Their function is to break English word order while keeping English vocabulary. The third is native-speaker-written template sets for three real languages: Chinese, Arabic and German. Because the world graph is shared, every question has an exact counterpart in every rendering. This gives a controlled cross-lingual multi-hop evaluation no natural benchmark offers. It also allows "mixed" worlds in which the bridging documents for one question are written in different languages. This is a multi-hop analogue of the generalized setting in MLQA, where question and context are in different languages.
The second component is the training matrix. Qwen3 models at 0.6B, 1.7B and 4B are trained with GRPO under a fixed budget of RL steps and sampled questions. The training data is English-only PhantomWiki in one condition, English plus rearranged English in another, and English plus the three native-template languages in a third. Supervised fine-tuning on the same synthetic data serves as a contrast condition. Held-out evaluation languages are never rendered in training: Hindi, Vietnamese and Spanish from MLQA, and Finnish, Russian, Korean, Bengali and Telugu from TyDi QA.
The third component is analysis, in two forms. One measures in which language the model writes its intermediate reasoning on target-language inputs, and whether that correlates with transfer. This bears on the hypothesis in Is continuous CoT better suited for multilingual reasoning? that latent reasoning is more language-agnostic. The other is an error audit that separates three failure types: a wrong bridge entity, a correct chain with a wrong final hop, and a correct answer produced in the wrong script or form.
Baselines
The naive reference point is the untrained base Qwen3 model prompted zero-shot on each evaluation set. Every gain is reported relative to it, separately per language.
The central competitor is the English-only GRPO recipe of Learning from Synthetic Data Improves Multi-hop Reasoning, reproduced exactly with its released synthetic data and evaluated zero-shot in the held-out languages. It is the strongest competitor because it is the only method in the file that produces real multi-hop gains from purely synthetic data. Its English result is the anchor: relative F1 gains of 56% to 131% across the four English benchmarks. No reported result was found on Lacuna for this or any method on non-English multi-hop QA, so the cross-lingual numbers are new measurements. Our variants receive the same model, number of RL steps, rollout count per question, reward function, and hyperparameter search budget. The only change is the renderer mix. Beating this baseline means a higher held-out-language F1, especially for verb-final languages, with no loss on English.
The natural-data competitor is the same GRPO run on real English HotpotQA training questions. Kabra et al. note that real in-domain data still beats synthetic transfer in English, so this condition tells us whether natural English text carries more or less English-specific bias than templates do.
The prompting competitor is cross-lingual QA prompting from Cross-lingual QA: A Key to Unlocking In-context Cross-lingual Performance. It reports 41.70 F1 on XQuAD with XGLM-7B, against 38.08 for target-language prompting. We run it on our Qwen3 base with the same number of demonstrations.
The translate-test competitor translates target-language inputs into English and answers them with the English GRPO model.
The key ablation compares rearranged English against native-template languages at matched training volume. This isolates whether breaking word order is the active ingredient or whether real target-language text is needed.
Why not just…?
Why not just render PhantomWiki directly in every target language? For languages where someone will write templates, this is the obvious move, and it is a real threat. It may simply win. That is why native-template rendering is one of our conditions rather than something we argue against. The point of interest is held-out languages, where no templates exist. There, the experiment tells us whether rearranged English recovers most of what native rendering gives. If native rendering beats rearranged English by a wide margin even on languages it never covered, then the useful ingredient is multilingual surface diversity, not word order, and we will report that.
Why not just translate the test questions into English (translate-test)? Riabi et al. found that their synthetic augmentation consistently outperformed the English-as-pivot pipeline on single-hop QA. Zero-shot Reading Comprehension by Cross-lingual Transfer Learning likewise found machine translation worse than multilingual representations. Multi-hop questions plausibly compound translation errors, because a mistranslated bridge entity breaks the whole chain. We do not assume this. Translate-test is run as a baseline, and the error audit counts bridge-entity failures caused by translation.
Why not just do supervised fine-tuning on the synthetic traces? Kabra et al. report that supervised fine-tuning overfits to the synthetic templates and fails to transfer in English. We expect this to be worse across languages, since copying templates is copying English word order. Our supervised fine-tuning condition tests whether the gap between reinforcement learning and supervised training widens with typological distance.
Why not just fine-tune on English SQuAD or HotpotQA and rely on the multilingual pretraining? MLQA reports roughly a 40% Exact Match drop from English for XLM trained this way, with 54.8 F1 in Arabic and 48.8 in Hindi. Riabi et al. report MiniLM at only 26.0 EM on MLQA. The HotpotQA-GRPO condition measures whether real English multi-hop data suffers the same fate.
Proposed Experiments and Decisive Checks
Evaluation runs on three layers. The first is the controlled layer: PhantomWiki worlds rendered in held-out conditions, where answers are exact and every question has a parallel version in each rendering. The second is a natural multi-hop layer. Since no multilingual multi-hop benchmark appears in the file, we will translate several hundred HotpotQA and 2WikiMultihopQA development questions into Hindi, Vietnamese, Spanish, Korean and Russian by machine translation followed by native-speaker correction. We acknowledge translation artifacts and report agreement between post-editors. The third is a single-hop guardrail layer that checks the curriculum does not damage basic reading: MLQA, XQuAD, and TyDiQA-GoldP, the last because its questions were written natively rather than translated. The metrics are answer F1 and Exact Match per language and per hop count, plus a transfer ratio defined as target-language gain divided by English gain.
Three outcomes would be decisive. If the English-only GRPO model's transfer ratio is near one across all languages, the skill is language-independent, and the word-order intervention is unnecessary. That would be a clean, useful negative result. If the transfer ratio falls specifically for verb-final languages, and rearranged-English training restores it while matched extra English training does not, word order is the binding variable, and the sandbox fixes it for free. If hop-count generalization (training on 2–3 hops, testing up to 9) holds in English but collapses in held-out languages, composition and language are entangled more deeply than surface order. We would then examine the reasoning-language and error audits to locate where the chain breaks.
References
- Learning from Synthetic Data Improves Multi-hop Reasoning
- Learning Disentangled Semantic Representations for Zero-Shot Cross-Lingual Transfer in Multilingual Machine Reading Comprehension
- Synthetic Data Augmentation for Zero-Shot Cross-Lingual Question Answering
- PAXQA: Generating Cross-lingual Question Answering Examples at Training Scale
- Pre-training Cross-lingual Open Domain Question Answering with Large-scale Synthetic Supervision
- MLQA: Evaluating Cross-lingual Extractive Question Answering
- Is continuous CoT better suited for multilingual reasoning?
- Cross-lingual QA: A Key to Unlocking In-context Cross-lingual Performance
- Zero-shot Reading Comprehension by Cross-lingual Transfer Learning with Multi-lingual Language Representation Model