Cross-Lingual Transfer of Logical Reasoning Capabilities

20 concepts13 papers2 proposals

When we train a large language model (LLM) to solve a complex mathematical word problem or execute multi-step logic, we often assume the model is learning both the language and the logic simultaneously. However, a deeper, more elegant architecture is at play: logical reasoning patterns are fundamentally language-independent. This insight forms the foundation of cross-lingual reasoning transfer—the phenomenon where training a model to reason in one language unlocks or enhances its ability to reason in entirely different languages.

The Conceptual Scaffold

Why does a model trained primarily on a high-resource language like English excel at reasoning in low-resource languages? The answer lies in conceptual scaffolding. High-resource languages provide a data-rich environment where models can internalize complex logical structures without being bottlenecked by vocabulary or syntactic scarcity.

As shown by Kritarth Prasad et al., parallel datasets in a high-resource language allow the model to use that language as a scaffold, learning how to solve and express mathematical problems in a target lower-resource language far more effectively than training on the low-resource language in isolation. In fact, Shahriar Kabir Nahin et al. demonstrate that exclusively training on a target low-resource language can limit a model's logical reasoning capabilities; introducing high-quality data from high-resource languages is essential to spark cross-lingual transfer.

This scaffolding effect means we do not need massive, native reasoning datasets for every language. Se June Joo et al. found that translating a relatively small subset of rationale-based training data (such as chain-of-thought prompts) can activate logical problem-solving in low-resource languages, yielding up to a tenfold performance improvement.

Similarly, Guangzhi Sun et al. and Amaru Cuba Gyllensten et al. show that mixing a high ratio of high-resource language data with a smaller portion of low-resource data allows models to inherit robust reasoning capabilities without degrading their core performance. Even when direct target-language instruction data is entirely absent, Hongbin Zhang et al. demonstrate that reasoning can be transferred effectively through English-only instruction tuning following a basic language alignment phase.

Shared Circuits and Information Routing

To understand why this transfer happens mechanically, we must look at how knowledge and logic are organized inside a transformer. Intuitively, one might think a multilingual model separates its English logic from its Hindi or Swedish logic. In reality, the model maps these concepts into a shared, language-agnostic representational space.

Debanjana Kar et al. illustrate this in document-level information extraction, showing that training on a multilingual corpus allows a model to develop language-agnostic strategies for aggregating information. The underlying facts and logical rules are stored centrally.

If the logic is shared, why do models often fail at complex tasks in low-resource languages? Chen Sun et al. uncovered that factual knowledge is stored in shared, language-agnostic neurons, but reasoning failures occur because the model's attention heads fail to route that information correctly in lower-resource contexts. By surgically repairing these misaligned attention heads in just one language, the authors achieved positive cross-lingual transfer across the model's entire vocabulary.

This proves that the logical engine is already there; cross-lingual fine-tuning simply fixes the routing mechanisms so the model can access it. This explains why fine-tuning on translated task-specific datasets, as explored by Chen Qiu 0005 et al., leads to highly consistent reasoning performance across multiple distinct languages.

The Cross-Lingual Synergy Bonus

Perhaps the most surprising finding in this domain is that cross-lingual transfer is not a one-way street. Training a model on non-English or multilingual data does not just help low-resource languages—it actively improves the model's reasoning capabilities in the high-resource language as well.

Nuo Chen et al. discovered that training a model on mathematical word problems across a diverse set of languages significantly boosted its math performance in English. Why? Because linguistic variety prevents the model from simply memorizing surface-level textual patterns. Instead, it forces the model to internalize the underlying, language-independent principles of logic.

Bryan Li echoes this paradigm, arguing that incorporating non-English data helps models develop more generalized logical reasoning capabilities that transcend specific linguistic boundaries. This synergy extends to multi-modal and multi-turn settings as well; Minbin Huang et al. found that training on bilingual datasets enhances general reasoning and instruction-following even for tasks executed entirely in a single language.

Taking this a step further, Shulin Huang et al. demonstrated that training reinforcement learning (RL) models on linguistically distant or grammatically complex non-English data yields better global performance than training on English alone. The complexity of the non-English data forces the model to explore more diverse and robust reasoning paths during its sampling phase, resulting in a highly generalized logical engine that performs better across all languages.

Key Foundational Work

The CoT Collection: Improving Zero-shot and Few-shot Learning of Language Models via Chain-of-Thought Fine-Tuning (2023) — contributed 2 concept elements to this direction.
ArgGen: Prompting Text Generation Models for Document-Level Event-Argument Aggregation (2022) — contributed 1 concept elements to this direction.
Multilingual Multimodal Learning with Machine Translated Text (2022) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_cross_lingual_reasoning_transfer_multili_d01988c2,
  title = {cross-lingual reasoning transfer (multilingual supervised fine-tuning)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/cross-lingual-reasoning-transfer-multilingual-supervised-fine-tuning/rdr_d01988c29ceb44c4b6a10ac39a33a9ea},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_cross_lingual_reasoning_transfer_multili_d01988c2_v1,
  title = {cross-lingual reasoning transfer (multilingual supervised fine-tuning)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/cross-lingual-reasoning-transfer-multilingual-supervised-fine-tuning/rdr_d01988c29ceb44c4b6a10ac39a33a9ea/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake