When interacting with state-of-the-art multilingual language models, a striking paradox emerges: a model can accurately comprehend a complex prompt written in a low-resource language or regional dialect, yet it frequently fails to generate a coherent, grammatically correct response in that same language. Instead, it often defaults to English or produces text marred by structural errors and hallucinations.
This phenomenon is known as translation direction asymmetry or cross-lingual performance asymmetry. It represents a fundamental imbalance in how large language models (LLMs) and neural machine translation (NMT) systems process language. Models consistently perform better when translating from a low-resource language into a high-resource language (many-to-one) than they do when translating from a high-resource language into a low-resource one (one-to-many).
To understand why this works, we must look at the geometry of a model's internal representation space and the differing mechanical demands of language encoding versus language decoding.
The Comprehension-Production Paradox
The performance chasm between understanding a language and generating it has been documented across a wide array of benchmarks. Evaluating ChatGPT, Bang et al. observed that in low-resource linguistic contexts, models accurately identify semantic features like sentiment but fail to produce fluent text in those same languages. Similarly, Gupta et al. introduced the IndicGenBench benchmark and revealed that models achieve relatively high comprehension scores when responding in English to regional queries, but struggle significantly when required to generate native-script summaries.
This directional bias is highly visible in machine translation. As Goyal et al. demonstrated with the FLORES-101 benchmark, multilingual models excel at many-to-one translation into English but degrade sharply when translating out of English. The same asymmetry appears in specialized domains: Tao et al. found that code-generation models translate low-resource programming languages into Python far better than the reverse, and Kanojia et al. showed that translation quality estimation is significantly more accurate when English is the target rather than the source.
The Geometry of Pretraining Bias
The root cause of this asymmetry lies in the highly skewed distribution of pretraining data. Because massive web-scraped corpora are overwhelmingly dominated by English, a model's internal activation space develops an English-centric topology.
As Vilar et al. argue, this data disproportion means that a model's internal representations are inherently more robust for structuring and generating English text. We can think of this geometrically: the conceptual space for English is dense, highly mapped, and continuous, whereas the space for a low-resource language is sparse and fractured.
Many-to-One (Easy) One-to-Many (Hard)
[Low-Resource Source] [High-Resource Source]
│ │
▼ ▼
(Map to Dense Space) (Map to Sparse Space)
│ │
▼ ▼
[Dense English Target] [Sparse Low-Resource Target]
When performing semantic retrieval or many-to-one translation, the model maps sparse input tokens from a low-resource language into its dense, well-defined English conceptual space. Zhang demonstrated that this query-to-target mapping is highly effective because the destination space is stable.
Conversely, when translating from English into a low-resource language, the model must project a dense concept onto a sparse token space. Gao et al. point out that standard target-side embeddings treat related words in low-resource languages as entirely distinct entities, failing to facilitate effective knowledge transfer during generation.
This creates a monotonic, unidirectional flow of semantic influence. As Xu et al. discovered, dominant conceptual vectors extracted from high-resource languages can easily steer model behavior in low-resource contexts, but conceptual vectors extracted from low-resource languages lack the robustness to control high-resource outputs.
Why Decoding is Harder than Encoding
Beyond representation density, the autoregressive generation process itself acts as a bottleneck. Language encoding (comprehension) is inherently more resilient to data scarcity and capacity dilution than language decoding (production).
Cahyawijaya et al. demonstrated that generating text in a low-resource language is significantly more sensitive to pretraining data volume than processing that same language as input. When encoding an input, a model can tolerate missing vocabulary or unfamiliar syntax by leveraging surrounding contextual cues to compress the text into a semantic vector. However, during the decoding phase, the model must make precise, token-by-token choices. It must master the exact morphological inflections, script requirements, and syntactic rules of the target language.
When a single model's parameter capacity is shared across hundreds of languages, this decoding proficiency is the first thing to degrade. Wenzek et al. found that as the number of supported languages increases, the language encoding component remains highly stable, whereas the decoding component suffers massive quality loss. The model retains the capacity to compress and understand diverse inputs, but its shared parameters cannot sustain the distinct generation mechanics required for multiple low-resource or morphologically rich target scripts.
Parallels in Human Cognition and Dialects
This computational asymmetry strongly mirrors patterns observed in human linguistics and dialectal variations. In second-language acquisition, Zhao highlights a persistent gap between receptive and productive knowledge: language learners routinely demonstrate the ability to comprehend and translate idioms back into their native language long before they possess the productive mastery required to generate those same idiomatic expressions accurately in the target language.
A similar split occurs when models encounter non-standard dialects. Kantharuban et al. quantified the "dialect gap" across language models, revealing that while models possess a high capacity for dialect comprehension, their performance drops precipitously when they are forced to generate text in a minority dialect rather than a standard one.
Designing Around the Asymmetry
Recognizing that translation directionality is fundamentally asymmetric has led researchers to move away from uniform training strategies.
Instead of treating many-to-one and one-to-many tasks identically, recent architectures exploit these directional dynamics. For example, Zheng et al. discovered that during the post-training phase of LLMs, mapping multiple languages into a single target causes severe linguistic interference and conflict. Conversely, mapping from a single source into multiple target languages fosters cross-lingual synergy. By using separate, specialized experts for many-to-one pathways while sharing parameters for one-to-many pathways, modern systems can mitigate capacity dilution.
Concurrently, targeted vocabulary adaptation and massive multilingual continue-pretraining, such as the EMMA-500 framework introduced by Ji et al., focus explicitly on reinforcing target-side representations. By expanding the model's token vocabulary and density in the target script, researchers aim to bridge the chasm between what a model can understand and what it can fluently produce.