Abstract
Multilingual large language models (MLLMs) often struggle to match their dominant-language performance when answering questions in non-dominant languages. While "translate-then-answer" methods offer a partial solution, the underlying mechanisms of their success—and their frequent failures in technical domains—have remained poorly understood. This paper introduces Native Language Prompting (NatLan), a framework that uses human cognitive analogies—Language Triggers (LT) and Domain Triggers (DT)—to interpret and improve multilingual knowledge elicitation. By employing a Multi-MLLM collaboration strategy with a domain-specific translator, NatLan achieves up to a 31.28% improvement in accuracy across five major languages, ensuring that critical technical terminology is retained during the translation process.
Introduction: The Multilingual Performance Gap
Despite being trained on vast multilingual corpora, state-of-the-art models like GPT-4o or Llama-3 exhibit a "native language" bias, performing significantly better in English (their dominant training language) than in others. A common workaround is to translate non-English queries into English before processing. However, this approach often stumbles on specialized or technical content. Standard translation tools frequently prioritize fluency over technical precision, replacing specific jargon with generic synonyms. This loss of precision prevents the model from accessing the specialized knowledge it possesses in its dominant language.
Conceptual Framework: Language and Domain Triggers
To address this, we propose two key concepts:
- Language Trigger (LT): The provision of a query in the model's dominant language, which reduces the "cognitive load" required for basic comprehension.
- Domain Trigger (DT): The retention of specific, fine-grained technical terms that act as keys to unlock specialized knowledge domains within the model's parameters.
As illustrated in Figure 1, standard Neural Machine Translation (NMT) might provide a sufficient LT but fail to retain the DT. For example, a chemistry question regarding "complete combustion" might be translated generically as "burned," leading the model to miss the technical nuances required for a correct answer.
Methodology: The NatLan Framework
NatLan implements a Multi-MLLM collaboration strategy to ensure both LT provision and DT retention. The architecture consists of two primary components:
- Translator LLM: A role-enhanced, domain-specific model (e.g., GPT-4o-mini or Qwen) that uses specialized instructions and few-shot examples to translate queries while strictly preserving technical DTs.
- Speaker LLM: The target model (e.g., Phi-3, Gemma, Mistral) that receives the high-quality English translation and generates the final response.
Experimental Evaluation
The researchers evaluated NatLan across five languages—Arabic, Chinese, French, German, and Japanese—using two comprehensive benchmarks:
- Multilingual MMLU (MMMLU): 14,079 multiple-choice questions spanning 57 disciplines.
- C-Eval: A Chinese-specific suite with 13,948 questions across 52 disciplines.
Key Results and Insights
NatLan consistently outperformed baselines such as "Self-Translation" and "Google-MT." Key findings include:
- Accuracy Improvements: NatLan delivered accuracy gains of up to 31.28% compared to direct answering in the original language.
- Superior DT Retention: In up to 87% of cases, NatLan provided better or comparable retention of domain-specific triggers compared to state-of-the-art NMT.
- Knowledge Activation Alignment: Analysis of the Speaker LLM's internal hidden states revealed that NatLan's translations caused internal activations to more closely mirror those seen when the model answers human-written "gold standard" English questions. This suggests that NatLan effectively "rectifies" the model's internal knowledge retrieval process.
Limitations and Future Directions
While highly effective, NatLan faces challenges in disciplines where translation may introduce errors, such as Mathematics (formula corruption) or culturally specific subjects where the relevant knowledge may not exist in the English training set. Future work will explore adaptive language switching to dynamically determine the optimal language for eliciting specific knowledge.
Conclusion
NatLan demonstrates that the key to unlocking the full potential of multilingual LLMs lies not just in translation, but in the strategic retention of domain-specific triggers. By aligning machine prompting with human cognitive principles, NatLan provides a robust framework for high-precision multilingual knowledge elicitation.
Summary updated