Directional Asymmetry in Multilingual Language Models

This direction is merged into another direction: translation direction asymmetry. Its last version stays readable and citable here.

Version 1 of 168 papers89 conceptsfrozen 2026-10-04

In an ideal multilingual system, the ability to translate between two languages should be symmetric: if a model can translate English into Swahili, it should be equally capable of translating Swahili into English. However, research across dozens of recent papers reveals a persistent "directional asymmetry." Models consistently perform better when translating into high-resource languages (like English) than when translating out of them.

This phenomenon isn't just a quirk of benchmarking; it reveals a fundamental split in how large language models (LLMs) process and generate human language.

The Comprehension-Generation Gap

The most striking evidence for this asymmetry comes from the "Translation Barrier Hypothesis." As proposed by Niyati Bafna, Daniel Khashabi, and colleagues in The Translation Barrier Hypothesis: Multilingual Generation with Large Language Models Suffers from Implicit Translation Failure, models often possess the "knowledge" to solve a task in a low-resource language but fail at the final step of expressing that knowledge.

The researchers argue that LLMs operate via an implicit two-stage pipeline:

  1. Comprehension: The model maps the source text (even in a low-resource language) into a language-agnostic intermediate representation.
  2. Generation: The model maps that representation into the target language.

When the target is a low-resource language, the model fails at the second stage. This is why, as Ife Adebara notes in the SAHARA benchmark, models can often interpret African languages effectively but struggle to generate fluent text in them. The "difficulty" of a language is localized in the output representation rather than the input understanding, a sentiment echoed by Ada Wan in Fairness in Representation for Multilingual NLP.

Why High-Resource Targets Win

The dominance of English in pre-training data creates a "gravitational pull" that favors English as a target. Wenhao Zhu, Hongyi Liu, and others found that GPT-4 and similar models excel at many-to-English translation because they can leverage their massive internal priors for English grammar and semantics to "clean up" the output (Multilingual Machine Translation with Large Language Models).

However, this same strength becomes a liability in the reverse direction. Duarte Miguel Alves and Ricardo Rei demonstrate that when translating from English into low-resource languages, these strong internal priors can actually override the source text, leading to fluent but completely hallucinated mistranslations (Hallucinations in Large Multilingual Translation Models).

Architectural Bottlenecks

The asymmetry is also baked into the way models share parameters. Ting-Rui Chiang and Graham Neubig found that in many-to-one translation, the decoder is highly efficient because it only has to master a single target distribution (Breaking Down Multilingual Machine Translation). In contrast, one-to-many translation forces the decoder to partition its limited capacity across many different linguistic structures, leading to a performance drop.

This "directional degeneration" is a significant hurdle. Tong Zheng and Bei Li point out that even when models are trained on symmetric bilingual data, they often prioritize target-side fluency over source-side accuracy, further widening the gap (Beyond English: Toward Inclusive and Scalable Multilingual Machine Translation with LLMs).

Asymmetry in Evaluation and Summarization

This imbalance extends beyond simple translation to complex reasoning tasks:

  • Summarization: Gyutae Park and Seojin Hwang show that LLMs can summarize foreign documents into English with high accuracy, but they struggle to generate summaries in low-resource languages from English sources (Low-Resource Cross-Lingual Summarization through Few-Shot Learning with Large Language Models).
  • Quality Estimation: Even when acting as a judge, a model's proficiency is directional. Diptesh Kanojia and Constantin Orasan found that LLMs are much better at estimating translation quality when the target language is high-resource (When LLMs Struggle: Reference-less Translation Evaluation for Low-resource Languages).

Restoring Balance

To overcome this, researchers are moving away from the assumption that scale alone fixes asymmetry. Muhammad Reza Qorib and Hwee Tou Ng argue that explicit training on bidirectional parallel pairs—rather than just more monolingual text—is essential to ensure functional symmetry (Just Go Parallel: Improving the Multilingual Capabilities of Large Language Models). Without these targeted interventions, multilingual models remain "receptive" polyglots: they can understand the world's languages, but they can only truly speak in a few.

Papers in this version

Showing 60 of 68 papers.

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_directional_asymmetry_in_multilingual_la_b8519f54,
  title = {Directional Asymmetry in Multilingual Language Models},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/directional-asymmetry-in-multilingual-language-models/rdr_b8519f54517e48a789d8bb8cca222354},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_directional_asymmetry_in_multilingual_la_b8519f54_v1,
  title = {Directional Asymmetry in Multilingual Language Models},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/directional-asymmetry-in-multilingual-language-models/rdr_b8519f54517e48a789d8bb8cca222354/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}