Modern large language models (LLMs) are remarkably polyglot, capable of processing and generating text across dozens of languages. Yet, beneath this surface fluency lies a deep cognitive asymmetry: a model's underlying reasoning, logic, and instruction-following capabilities remain heavily anchored to English. This creates a counter-intuitive phenomenon known as cross-lingual instruction bias, where a model performs significantly better on a non-English task when the instructions or prompts are written in English rather than the target language.
Why the Core Engine Favors English
To understand why this happens, we have to look at the pre-training distribution. Because the vast majority of web-scale training data is in English, the structural patterns of logical reasoning, step-by-step problem solving, and instruction adherence are carved deepest into the model's weights using English tokens.
As Viet Dac Lai et al. demonstrate in ChatGPT Beyond English: Towards a Comprehensive Evaluation of Large Language Models in Multilingual Learning, LLMs exhibit a systematic bias where their task-execution and internal reasoning capabilities are more effectively triggered by instructions in a dominant training language like English. This means that even when a model "knows" a language, its ability to think within that language is constrained.
For instance, when evaluating massive multitask language understanding in Arabic, Fajri Koto et al. found in ArabicMMLU: Assessing Massive Multitask Language Understanding in Arabic that models often achieve superior performance on non-English tasks when provided with English-language instructions and labels rather than native ones. Similarly, Haoqiang Kang et al. showed in Translate to Disambiguate: Zero-shot Multilingual Word Sense Disambiguation with Pretrained Language Models that prompting multilingual models in English frequently yields better results for source-language tasks because the dominant pre-training language provides the most reliable pathway to the model's internal reasoning engine.
Empirical Evidence Across Diverse Languages
This English-prompting advantage is not an isolated quirk of a few high-resource languages; it is a systemic pattern observed across low-resource languages and highly specialized domains:
- Persian and Ukrainian: In a study on Persian language models, Sara Baruni et al. discovered in Benchmarking Large Language Models for Persian: A Preliminary Study Focusing on ChatGPT that prompting in English often yields better results for Persian tasks than prompting in Persian itself. Similarly, for Ukrainian question-answering systems, Tiberiu Boros et al. observed in Fine-Tuning and Retrieval Augmented Generation for Question Answering Using Affordable Large Language Models that models demonstrate improved performance and instruction following when the prompts are provided in English.
- Kazakh and Spatio-Temporal Logic: When evaluating models on Kazakhstani regional knowledge, Mukhammed Togmanov showed in KazMMLU: Evaluating Language Models on Kazakh, Russian, and Regional Knowledge of Kazakhstan that logical processing capabilities are more effectively triggered by English instructions, even when the task content is natively non-English. This extends to industrial applications: Qiang Li et al. found in How LLMs React to Industrial Spatio-Temporal Data? Assessing Hallucination with a Novel Traffic Incident Benchmark Dataset that translating non-English context into English significantly boosts a model's reasoning performance, proving that the underlying logic engine operates more efficiently on English token patterns.
The Hidden Costs: Cultural Leakage and Language Reversion
While using English prompts acts as a powerful shortcut to unlock a model's reasoning, it introduces severe side effects. The language of the prompt does not just activate a neutral logical processor; it carries cultural and behavioral baggage that can distort the output.
Cultural Bias
English prompts can induce a Western cultural bias. In Does Mapo Tofu Contain Coffee? Probing LLMs for Food-related Cultural Knowledge, Li Zhou et al. found that the language used for instructions exerts more influence over a model's output than the language of the subject matter itself, leading to a Westernized interpretation of non-Western concepts when prompted in English.
Compounding this issue, advanced reasoning techniques like Chain-of-Thought (CoT) can backfire. Qianying Liu et al. noted in 7 Points to Tsinghua but 10 Points to ? Assessing Large Language Models in Agentic Multilingual National Bias that CoT prompting can inadvertently exacerbate underlying cultural biases by forcing the model to engage more deeply with language-specific training patterns, creating a conflict between reasoning depth and objective neutrality.
Language Reversion
English prompts frequently cause a model to forget to output in the target language. Songbo Hu et al. highlighted this in DIALIGHT: Lightweight Multilingual Development and Evaluation of Task-Oriented Dialogue Systems with Large Language Models, showing that system prompts in English create a significant linguistic bias that causes multilingual models to revert to English instead of the intended target language.
This failure in cross-lingual instruction following is especially pronounced in cognitively demanding multimodal tasks, as observed by Sara Papi et al. in MCIF: Multimodal Crosslingual Instruction-Following Benchmark from Scientific Talks, where advanced models default to English during complex tasks like summarization despite explicit instructions to output in other languages.
How Post-Training and Architecture Reinforce the Anchor
Why does this bias persist even after models undergo multilingual alignment? The answer lies in how post-training and internal mechanics operate.
Surprisingly, alignment techniques often worsen the disparity. Nikhil Sharma et al. discovered in Faux Polyglot: A Study on Information Disparity in Multilingual Large Language Models that post-training processes like instruction tuning can shift a model's linguistic preferences toward dominant languages like English, overriding the base model's native tendencies.
Mechanistically, English acts as a "default" state. In a study on machine translation, Théo Lasnier et al. found in Translation Heads: Disentangling meaning from language in LLM-based machine translation that generating non-default languages requires continuous active suppression of a primary linguistic bias; when these steering mechanisms are inhibited, the model reverts to its dominant training language.
This asymmetry also affects how knowledge is stored and updated. Jiaan Wang et al. demonstrated in Cross-Lingual Knowledge Editing in Large Language Models a significant "cross-lingual gap" in model editing: updating a fact in English is highly effective because the internal representations of the dominant language are robust, whereas editing the same fact in secondary languages often fails to propagate across linguistic boundaries.
Navigating the Bias: Strategic Language Selection
Can we exploit or mitigate this cross-lingual bias? The optimal prompting strategy depends heavily on the model's lineage and the task at hand.
In Benchmarking Chinese Commonsense Reasoning of LLMs: From Chinese-Specifics to Reasoning-Memorization Correlations, Jiang Wu et al. point out that while English-centric models benefit from cross-lingual prompting (instructing in English for a native task), models specialized and pre-trained in other languages achieve better reasoning performance using native-language Chain-of-Thought techniques.
Interestingly, the interaction between languages can sometimes be leveraged counter-intuitively to break default biases. Shabnam Behzad et al. discovered in To Ask LLMs about English Grammaticality, Prompt Them in a Different Language that prompting a model in a non-English language can actually improve its accuracy at judging English grammar. By shifting the instructional language, the prompt breaks the model's default high-resource behaviors and "sensitizes" it to linguistic structure, demonstrating that cross-lingual prompting can occasionally be engineered to mitigate inherent training biases.
Key Foundational Work
Polyglot Prompt: Multilingual Multitask Prompt Training (2022) — contributed 1 concept elements to this direction.
How many data points is a prompt worth? (2021) — contributed 1 concept elements to this direction.