The Knowledge-Language Decouple in Multilingual Models

This direction is split into other directions: language-dependent knowledge retrieval. Its last version stays readable and citable here.

Version 1 of 151 papers76 conceptsfrozen 2026-10-04

If you wanted to learn about the specific nuances of a traditional Ethiopian wedding or a regional Arabic dialect, you might assume that asking a Large Language Model (LLM) in the native language of that culture would yield the most accurate results.

However, recent research reveals a "native language paradox": multilingual models often possess more factual knowledge about a specific country when queried in English than when queried in that country’s own native language. This phenomenon, often called the Knowledge-Language Decouple, suggests that cultural information is frequently "trapped" within high-resource language representations and fails to transfer to the model's understanding of other languages.

The Native Language Paradox

The core of this issue is a disconnect between linguistic proficiency and factual retrieval. In GeoMLAMA: Geo-Diverse Commonsense Probing on Multilingual Pre-Trained Language Models, Da Yin and colleagues found that larger models aren't necessarily more culturally aware. Instead, they discovered that high-resource languages like English are often more effective at eliciting region-specific cultural knowledge than the native languages of those regions.

This isn't just a matter of the model being "smarter" in English. As Eshaan Tanwar demonstrates in Do You Know About My Nation? Investigating Multilingual Language Models’ Cultural Literacy Through Factual Knowledge, models exhibit a fundamental decoupling where the internal knowledge hierarchy remains lopsided toward Western defaults. Even when a model has been exposed to cultural facts during pre-training, the inherent features of a low-resource language can act as a barrier, preventing the model from accessing that information during inference.

Language as a Retrieval Key

To understand why this happens, we must view language not just as a medium of communication, but as a retrieval mechanism.

In Global Gallery: The Fine Art of Painting Culture Portraits through Multilingual Instruction Tuning, Anjishnu Mukherjee and Antonios Anastasopoulos argue that because base models are disproportionately optimized for English, English becomes the most reliable "key" for unlocking internal knowledge. When a model learns about a non-Western festival through an English Wikipedia article, that knowledge is indexed under English linguistic triggers. If the model is later asked about that festival in a low-resource language, it may fail to find the "path" to that information, leading to hallucinations or generic, Western-biased responses.

This "siloing" of knowledge is further explored by Lajanugen Logeswaran in Understanding the Capabilities and Limitations of Large Language Models for Cultural Commonsense. Their work shows that models often correctly identify cultural facts in English but fail in the native language, indicating that cross-lingual knowledge transfer is inconsistent. The knowledge is present in the weights, but the "linguistic interface" is broken.

Benchmarking the Cultural Blind Spots

To measure this gap, researchers have developed specialized benchmarks that move beyond simple translation.

  • BLEnD: Dimosthenis Antypas and Nayeon Lee introduced BLEnD to test "everyday" knowledge across 16 countries. They found GPT-4 performs up to 57% better on Western cultural knowledge than on regions like Ethiopia.
  • ArabCulture: Abdelrahman Boda Sadallah showed in Commonsense Reasoning in Arab Culture that even state-of-the-art models struggle with region-specific nuances, proving that simply translating English benchmarks is inadequate for capturing true cultural competence.
  • BertaQA: Julen Etxaniz and Mikel Artetxe used BertaQA to show that while translation-based inference helps access global knowledge, it fails when the underlying local facts were never integrated into the model's weights during training.

Activating Latent Knowledge

If the knowledge is there but inaccessible, how do we "switch" it on?

One promising direction is code-switching—the practice of mixing languages. Seoyeon Kim demonstrates in Can Code-Switched Texts Activate a Knowledge Switch in LLMs? that prompting models with a mix of English and a native language (like Korean) can act as a "knowledge switch." This hybrid approach provides the model with the English retrieval keys it needs while maintaining the cultural context of the native language, resulting in more accurate and nuanced retrievals than English-only or translated prompts.

Another approach, CulFiT by Ren Feng and Shen Gao, attempts to bridge the gap during fine-tuning. By using multilingual critique data and rewarding the model for "atomic knowledge units" specific to a culture, they can reduce the "language inconsistency" where a model provides a culturally accurate response in one language but reverts to a biased one in English.

Why Translation is Not Enough

A common pitfall in the field has been the reliance on translated datasets for instruction tuning. Pinzhen Chen and Barry Haddow argue in Is It Good Data for Multilingual Instruction Tuning or Just Bad Multilingual Evaluation for Large Language Models? that this practice creates a performance bottleneck. Models trained on translated data often retain the cultural and factual context of the source language (usually English), which impairs their ability to perform tasks requiring genuine local expertise.

True cultural alignment requires more than just a "linguistic mask"; it requires ensuring that the model's internal knowledge is grounded in the local vernacular and accessible across all the languages it speaks.

Papers in this version

History & citationVersion 1 · 2026-10-04

Split into other directions · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_the_knowledge_language_decouple_in_multi_4884c9a5,
  title = {The Knowledge-Language Decouple in Multilingual Models},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/the-knowledge-language-decouple-in-multilingual-models/rdr_4884c9a53c904f7f93be37cce8ae05cd},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 1, frozen text):

@misc{lacuna_the_knowledge_language_decouple_in_multi_4884c9a5_v1,
  title = {The Knowledge-Language Decouple in Multilingual Models},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/the-knowledge-language-decouple-in-multilingual-models/rdr_4884c9a53c904f7f93be37cce8ae05cd/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}