When we train a large language model (LLM) to solve a complex mathematical word problem or execute multi-step logic, we often assume the model is learning both the language and the logic simultaneously. However, a deeper, more elegant architecture is at play: logical reasoning patterns are fundamentally language-independent. This insight forms the foundation of cross-lingual reasoning transfer—the phenomenon where training a model to reason in one language unlocks or enhances its ability to reason in entirely different languages.
The Conceptual Scaffold
Why does a model trained primarily on a high-resource language like English excel at reasoning in low-resource languages? The answer lies in conceptual scaffolding. High-resource languages provide a data-rich environment where models can internalize complex logical structures without being bottlenecked by vocabulary or syntactic scarcity.
As shown by Kritarth Prasad et al., parallel datasets in a high-resource language allow the model to use that language as a scaffold, learning how to solve and express mathematical problems in a target lower-resource language far more effectively than training on the low-resource language in isolation. In fact, Shahriar Kabir Nahin et al. demonstrate that exclusively training on a target low-resource language can limit a model's logical reasoning capabilities; introducing high-quality data from high-resource languages is essential to spark cross-lingual transfer.
This scaffolding effect means we do not need massive, native reasoning datasets for every language. Se June Joo et al. found that translating a relatively small subset of rationale-based training data (such as chain-of-thought prompts) can activate logical problem-solving in low-resource languages, yielding up to a tenfold performance improvement.
Similarly, Guangzhi Sun et al. and Amaru Cuba Gyllensten et al. show that mixing a high ratio of high-resource language data with a smaller portion of low-resource data allows models to inherit robust reasoning capabilities without degrading their core performance. Even when direct target-language instruction data is entirely absent, Hongbin Zhang et al. demonstrate that reasoning can be transferred effectively through English-only instruction tuning following a basic language alignment phase.
Shared Circuits and Information Routing
To understand why this transfer happens mechanically, we must look at how knowledge and logic are organized inside a transformer. Intuitively, one might think a multilingual model separates its English logic from its Hindi or Swedish logic. In reality, the model maps these concepts into a shared, language-agnostic representational space.
Debanjana Kar et al. illustrate this in document-level information extraction, showing that training on a multilingual corpus allows a model to develop language-agnostic strategies for aggregating information. The underlying facts and logical rules are stored centrally.
If the logic is shared, why do models often fail at complex tasks in low-resource languages? Chen Sun et al. uncovered that factual knowledge is stored in shared, language-agnostic neurons, but reasoning failures occur because the model's attention heads fail to route that information correctly in lower-resource contexts. By surgically repairing these misaligned attention heads in just one language, the authors achieved positive cross-lingual transfer across the model's entire vocabulary.
This proves that the logical engine is already there; cross-lingual fine-tuning simply fixes the routing mechanisms so the model can access it. This explains why fine-tuning on translated task-specific datasets, as explored by Chen Qiu 0005 et al., leads to highly consistent reasoning performance across multiple distinct languages.
The Cross-Lingual Synergy Bonus
Perhaps the most surprising finding in this domain is that cross-lingual transfer is not a one-way street. Training a model on non-English or multilingual data does not just help low-resource languages—it actively improves the model's reasoning capabilities in the high-resource language as well.
Nuo Chen et al. discovered that training a model on mathematical word problems across a diverse set of languages significantly boosted its math performance in English. Why? Because linguistic variety prevents the model from simply memorizing surface-level textual patterns. Instead, it forces the model to internalize the underlying, language-independent principles of logic.
Bryan Li echoes this paradigm, arguing that incorporating non-English data helps models develop more generalized logical reasoning capabilities that transcend specific linguistic boundaries. This synergy extends to multi-modal and multi-turn settings as well; Minbin Huang et al. found that training on bilingual datasets enhances general reasoning and instruction-following even for tasks executed entirely in a single language.
Taking this a step further, Shulin Huang et al. demonstrated that training reinforcement learning (RL) models on linguistically distant or grammatically complex non-English data yields better global performance than training on English alone. The complexity of the non-English data forces the model to explore more diverse and robust reasoning paths during its sampling phase, resulting in a highly generalized logical engine that performs better across all languages.
Key Foundational Work
The CoT Collection: Improving Zero-shot and Few-shot Learning of Language Models via Chain-of-Thought Fine-Tuning (2023) — contributed 2 concept elements to this direction.
ArgGen: Prompting Text Generation Models for Document-Level Event-Argument Aggregation (2022) — contributed 1 concept elements to this direction.
Multilingual Multimodal Learning with Machine Translated Text (2022) — contributed 1 concept elements to this direction.