The English Anchor in Multilingual Instruction Following

45 concepts37 papers1 proposal

Modern large language models (LLMs) are remarkably polyglot, capable of processing and generating text across dozens of languages. Yet, beneath this surface fluency lies a deep cognitive asymmetry: a model's underlying reasoning, logic, and instruction-following capabilities remain heavily anchored to English. This creates a counter-intuitive phenomenon known as cross-lingual instruction bias, where a model performs significantly better on a non-English task when the instructions or prompts are written in English rather than the target language.

Why the Core Engine Favors English

To understand why this happens, we have to look at the pre-training distribution. Because the vast majority of web-scale training data is in English, the structural patterns of logical reasoning, step-by-step problem solving, and instruction adherence are carved deepest into the model's weights using English tokens.

As Viet Dac Lai et al. demonstrate in ChatGPT Beyond English: Towards a Comprehensive Evaluation of Large Language Models in Multilingual Learning, LLMs exhibit a systematic bias where their task-execution and internal reasoning capabilities are more effectively triggered by instructions in a dominant training language like English. This means that even when a model "knows" a language, its ability to think within that language is constrained.

For instance, when evaluating massive multitask language understanding in Arabic, Fajri Koto et al. found in ArabicMMLU: Assessing Massive Multitask Language Understanding in Arabic that models often achieve superior performance on non-English tasks when provided with English-language instructions and labels rather than native ones. Similarly, Haoqiang Kang et al. showed in Translate to Disambiguate: Zero-shot Multilingual Word Sense Disambiguation with Pretrained Language Models that prompting multilingual models in English frequently yields better results for source-language tasks because the dominant pre-training language provides the most reliable pathway to the model's internal reasoning engine.

Empirical Evidence Across Diverse Languages

This English-prompting advantage is not an isolated quirk of a few high-resource languages; it is a systemic pattern observed across low-resource languages and highly specialized domains:

The Hidden Costs: Cultural Leakage and Language Reversion

While using English prompts acts as a powerful shortcut to unlock a model's reasoning, it introduces severe side effects. The language of the prompt does not just activate a neutral logical processor; it carries cultural and behavioral baggage that can distort the output.

Cultural Bias

English prompts can induce a Western cultural bias. In Does Mapo Tofu Contain Coffee? Probing LLMs for Food-related Cultural Knowledge, Li Zhou et al. found that the language used for instructions exerts more influence over a model's output than the language of the subject matter itself, leading to a Westernized interpretation of non-Western concepts when prompted in English.

Compounding this issue, advanced reasoning techniques like Chain-of-Thought (CoT) can backfire. Qianying Liu et al. noted in 7 Points to Tsinghua but 10 Points to ? Assessing Large Language Models in Agentic Multilingual National Bias that CoT prompting can inadvertently exacerbate underlying cultural biases by forcing the model to engage more deeply with language-specific training patterns, creating a conflict between reasoning depth and objective neutrality.

Language Reversion

English prompts frequently cause a model to forget to output in the target language. Songbo Hu et al. highlighted this in DIALIGHT: Lightweight Multilingual Development and Evaluation of Task-Oriented Dialogue Systems with Large Language Models, showing that system prompts in English create a significant linguistic bias that causes multilingual models to revert to English instead of the intended target language.

This failure in cross-lingual instruction following is especially pronounced in cognitively demanding multimodal tasks, as observed by Sara Papi et al. in MCIF: Multimodal Crosslingual Instruction-Following Benchmark from Scientific Talks, where advanced models default to English during complex tasks like summarization despite explicit instructions to output in other languages.

How Post-Training and Architecture Reinforce the Anchor

Why does this bias persist even after models undergo multilingual alignment? The answer lies in how post-training and internal mechanics operate.

Surprisingly, alignment techniques often worsen the disparity. Nikhil Sharma et al. discovered in Faux Polyglot: A Study on Information Disparity in Multilingual Large Language Models that post-training processes like instruction tuning can shift a model's linguistic preferences toward dominant languages like English, overriding the base model's native tendencies.

Mechanistically, English acts as a "default" state. In a study on machine translation, Théo Lasnier et al. found in Translation Heads: Disentangling meaning from language in LLM-based machine translation that generating non-default languages requires continuous active suppression of a primary linguistic bias; when these steering mechanisms are inhibited, the model reverts to its dominant training language.

This asymmetry also affects how knowledge is stored and updated. Jiaan Wang et al. demonstrated in Cross-Lingual Knowledge Editing in Large Language Models a significant "cross-lingual gap" in model editing: updating a fact in English is highly effective because the internal representations of the dominant language are robust, whereas editing the same fact in secondary languages often fails to propagate across linguistic boundaries.

Can we exploit or mitigate this cross-lingual bias? The optimal prompting strategy depends heavily on the model's lineage and the task at hand.

In Benchmarking Chinese Commonsense Reasoning of LLMs: From Chinese-Specifics to Reasoning-Memorization Correlations, Jiang Wu et al. point out that while English-centric models benefit from cross-lingual prompting (instructing in English for a native task), models specialized and pre-trained in other languages achieve better reasoning performance using native-language Chain-of-Thought techniques.

Interestingly, the interaction between languages can sometimes be leveraged counter-intuitively to break default biases. Shabnam Behzad et al. discovered in To Ask LLMs about English Grammaticality, Prompt Them in a Different Language that prompting a model in a non-English language can actually improve its accuracy at judging English grammar. By shifting the instructional language, the prompt breaks the model's default high-resource behaviors and "sensitizes" it to linguistic structure, demonstrating that cross-lingual prompting can occasionally be engineered to mitigate inherent training biases.

Key Foundational Work

Polyglot Prompt: Multilingual Multitask Prompt Training (2022) — contributed 1 concept elements to this direction.
How many data points is a prompt worth? (2021) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_cross_lingual_instruction_bias_41c9f100,
  title = {cross-lingual instruction bias},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/cross-lingual-instruction-bias/rdr_41c9f100695c4e158c7d16c4b6e47bc5},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 1, frozen text):

@misc{lacuna_cross_lingual_instruction_bias_41c9f100_v1,
  title = {cross-lingual instruction bias},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/cross-lingual-instruction-bias/rdr_41c9f100695c4e158c7d16c4b6e47bc5/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake