Abstract
Multilingual large language models (MLLMs) do not perform as well when answering questions in non-dominant languages as they do in their dominant languages. Although existing translate-then-answer methods alleviate this issue, the mechanisms behind their effectiveness remain unclear. In this study, we analogize the dominant language of MLLMs to the native language of humans and use two human cognitive features: the Language Trigger (LT) and the Domain Trigger (DT), to interpret the mechanisms behind translate-then-answer methods. This reveals that while sufficient LTs are provided by these methods, there remains a deficiency in DT retention. To mitigate this issue, we propose Native Language Prompting (NatLan), employing a Multi-MLLM collaboration strategy and introducing an additional role-enhanced domain-specific MLLM with stronger multilingual understanding capabilities as the translator. Across five language QA benchmarks, NatLan achieves up to a 31.28% improvement in accuracy and, compared to existing state-of-the-art methods, provides comparable or greater retention of DTs in up to 87% of cases.
Introduction
Most state-of-the-art Multilingual Large Language Models (MLLMs) are fundamentally "English-centric." Training corpora are heavily skewed, with English tokens often accounting for over 70% of the data (e.g., Llama). This imbalance creates a significant performance gap: a model may possess the underlying knowledge to solve a complex physics problem in English but fail when presented with the same question in Swahili, Chinese, or French.
While "translate-then-answer" strategies have been used to bridge this gap, they often suffer from a subtle but critical flaw: Domain Trigger (DT) loss. Standard translation tools or model self-translations frequently replace precise technical jargon with generic terms. For instance, a specific chemical term like "completely combusted" might be simplified to "burned," failing to activate the high-density knowledge clusters the model holds in its dominant language.
The Conceptual Framework: LT and DT
To understand this phenomenon, we introduce two concepts: Language Triggers (LT) and Domain Triggers (DT).
- Language Triggers (LT): These reduce cognitive load by moving the problem into the model's native tongue (English).
- Domain Triggers (DT): These are specialized terms that narrow the search space within the model's parameters to locate exact factual information.
As shown in Figure 1, while standard translation provides the correct language context (LT), it often fails to provide the specific domain context (DT) required for a correct answer. NatLan is designed to preserve both.
Methodology: The NatLan Framework
We propose NatLan (Native Language Prompting), which utilizes a Role-Enhanced Multi-MLLM Collaboration strategy. Rather than relying on a single model to both translate and answer, we decouple these tasks:
- The Translator LLM: A model with superior multilingual capabilities (e.g., GPT-4o-mini) is assigned the role of a specialized translator. It is enhanced with domain-specific instructions and few-shot examples to ensure technical terminology is preserved.
- The Speaker LLM: The target model (e.g., Phi-3, Llama-2) receives this high-precision translation in its dominant language and provides the final answer.
Evaluation and Key Results
NatLan was evaluated across major benchmarks, including MMMLU (covering 57 disciplines in five languages) and C-Eval (a comprehensive Chinese evaluation suite).
Key findings include:
- Significant Accuracy Gains: NatLan achieved up to a 31.28% improvement in accuracy compared to direct answering in non-native languages.
- Superior DT Retention: The framework retained Domain Triggers in up to 87% of cases compared to standard Neural Machine Translation (NMT) methods.
- Activation Rectification: Internal analysis using t-SNE visualizations showed that NatLan causes the model's internal hidden states to align closely with its activations when answering "gold standard" English versions of the same questions.
Discussion and Limitations
NatLan demonstrates that translation is not just about linguistic understanding but about "unlocking" internal knowledge. However, limitations remain. In disciplines like Mathematics, where formulas are universal, translation can sometimes introduce errors. Additionally, some cultural nuances (e.g., specific folklore terms) are so deeply tied to a language that translating them into English can occasionally cause conflict with the model's training data.
Future work will explore adaptive language switching—automatically determining when a question requires high-precision translation and when it should be addressed directly in the source language.
Summary updated