If you ask a state-of-the-art multilingual language model about a highly localized cultural norm—such as Iranian social taboos, traditional Korean customs, or regional Indian geography—you might naturally assume that prompting it in the native language of that region would yield the most accurate response. However, recent evaluations reveal a striking contradiction known as the native language paradox: models frequently retrieve culture-specific facts more accurately when queried in English than in the culture's own native language.
This phenomenon highlights a fundamental cross-lingual knowledge retrieval gap. Multilingual capabilities do not inherently guarantee that a model's internal knowledge is equitably distributed or accessible across all the languages it supports. Instead, factual and cultural information often becomes trapped within specific linguistic silos.
The Native Language Paradox
As demonstrated by Da Yin et al. using the GeoMLAMA benchmark, high-resource languages like English act as a more effective mechanism for eliciting region-specific commonsense than the local vernacular spoken in those regions. This is not an isolated flaw but a systemic pattern across diverse cultures:
- Social Norms: In evaluations of Iranian social expectations, Hamidreza Saffari et al. found that models like GPT-4o fail to accurately classify local taboos when prompted in Farsi, despite demonstrating an understanding of those same concepts when queried in English.
- Everyday Knowledge: Using the BLEnD benchmark, Dimosthenis Antypas et al. showed that models perform up to 57% better on Western cultural knowledge than on underrepresented regions, and that local cultural facts are frequently encoded in the English pre-training data rather than the target language's data.
- Regional Contexts: Lajanugen Logeswaran et al. and Pritika Rohera et al. documented a persistent bottleneck where factual knowledge remains accessible in English but unreachable in lower-resource tongues, exposing a severe disconnect between factual grounding and linguistic context.
This creates a frustrating tradeoff for global applications: the most linguistically appropriate prompt for a user is often the least effective for extracting accurate information about their own culture, as noted by Anjishnu Mukherjee et al..
Why Knowledge Gets Siloed
The root cause of this retrieval gap lies in the asymmetry of cross-lingual pre-training. Because training corpora are overwhelmingly dominated by English, a vast amount of the world's localized cultural data is actually learned from English descriptions (e.g., global news or translated Wikipedia articles) rather than native text.
When a model processes this data, the factual information becomes structurally bound to high-resource linguistic tokens. This creates a localized "language trigger" effect. As Tuo Zhang et al. observe, cultural context can be tightly coupled to specific linguistic cues, preventing the model from generalizing that insight when prompted in a different language or multimodal setting.
Furthermore, cross-lingual transfer within these networks is fundamentally asymmetric. Guijin Son et al. demonstrate that abstract logic and reasoning capabilities transfer across languages with relative ease, but culture-specific factual knowledge does not. A model can use English-based reasoning to solve a complex puzzle, but it cannot easily map a native-language prompt to a factual weight that was optimized exclusively in an English context.
This results in what Bin Wang et al. describe as fragmented knowledge, where a model's internal world view varies wildly depending on the language of the prompt. While cross-lingual information retrieval can successfully aggregate global knowledge across languages, Benjamin Müller et al. note that systems remain highly dependent on specific language channels to surface localized expertise.
Disentangling Language from Concept
To diagnose and fix this gap, researchers must isolate whether a model fails because it completely lacks a cultural concept or because it cannot navigate the linguistic barrier to reach it.
[Native Language Query] ───(Concept Shift?)───► [Does the model understand the cultural concept?]
│
(Retrieval Gap?)
│
▼
[Can the model access it in this language?]
Anubha Kabra et al. introduced a technique called concept shift decomposition to isolate these errors. By comparing model performance on native-language metaphors against their machine-translated English counterparts, they proved that failures are often driven by a lack of cultural conceptual knowledge rather than a simple translation failure. If a model fails even on the English translation of a native metaphor, the issue is a missing concept; if it succeeds only on the English translation, the issue is a retrieval gap.
This distinction is critical when training models. Simon Yu investigated the common practice of using translated datasets for multilingual instruction tuning, finding that the primary performance bottleneck is a lack of language-specific cultural knowledge rather than the stylistic artifacts of translation ("translationese"). Models trained purely on translated English data retain the cultural context of the source language, leaving them ill-equipped for tasks requiring genuine local expertise.
Conversely, Julen Etxaniz et al. demonstrated via the BertaQA benchmark that the flow of knowledge can be bidirectional: continued pre-training in a low-resource language (Basque) can successfully transfer local knowledge back into high-resource language representations (English).
Strategies for Unlocking Trapped Knowledge
If the knowledge exists within the model's parameters but is merely inaccessible via native prompts, how can we retrieve it without expensive retraining? Researchers have proposed several algorithmic and prompting interventions to bridge this gap.
Language Representation Projection (LRP2)
Instead of modifying the model's weights or relying on text-level translation, Junzhuo Li et al. introduced LRP2. This parameter-free method mathematically projects non-English query representations into an "English-like" hidden space during the forward pass. By shifting the activation vectors into the region where the model's factual knowledge is most densely concentrated, LRP2 effectively unlocks the richer, English-encoded factual base directly from a native-language input.
Anthropological Prompting
To counteract the default Anglocentric bias of alignment tuning, Badr AlKhamissi et al. developed Anthropological Prompting. This technique structures the prompt to force the model to explicitly reason through a specific cultural lens before answering, encouraging the activation of latent cultural contexts that standard prompts fail to trigger.
Multilingual Feedback
When a model is forced to operate entirely within a low-resource language, it lacks the ability to verify its own outputs against its broader internal knowledge base. To solve this, Shangbin Feng et al. developed a multilingual feedback mechanism.
When queried, the model generates candidate answers and subsequently critiques them by generating reflections in multiple related, higher-resource languages. This cross-lingual cross-examination allows the model to tap into parameters that were locked under the initial prompt's language, significantly reducing hallucinations and improving its ability to abstain when it lacks local certainty.
Key Foundational Work
GeoMLAMA: Geo-Diverse Commonsense Probing on Multilingual Pre-Trained Language Models (2022) — contributed 2 concept elements to this direction.
Language Representation Projection: Can We Transfer Factual Knowledge across Languages in Multilingual Language Models? (2023) — contributed 1 concept elements to this direction.
Not Enough Data to Pre-train Your Language Model? MT to the Rescue! (2023) — contributed 1 concept elements to this direction.