Document Level Machine Translation and Discourse Coherence

This direction is split into other directions: discourse-level inconsistency. Its last version stays readable and citable here.

Version 1 of 1335 papers410 conceptsfrozen 2026-10-04

For decades, the gold standard for Machine Translation (MT) was the isolated sentence. Systems like Google's Neural Machine Translation achieved remarkable accuracy by treating every sentence as an independent island. However, as Samuel Läubli and Rico Sennrich demonstrated in Has Machine Translation Achieved Human Parity?, this "sentence-level" success is deceptive. When professional translators evaluate full documents, they consistently find machine output disjointed, proving that true human parity requires a grasp of discourse coherence.

The Tunnel Vision Problem

The fundamental challenge is that many linguistic choices depend on information outside the current sentence. Researchers often describe this as a "tunnel vision" problem. When a model translates a sentence in isolation, it lacks the "box cover" of the puzzle, leading to several specific failures:

  1. Anaphora and Coreference: A model might translate "it" as a masculine pronoun in one sentence and a feminine one in the next, simply because the antecedent (the noun "it" refers to) appeared three sentences ago.
  2. Lexical Consistency: Technical terms or proper nouns might be translated using different synonyms across a document, confusing the reader.
  3. Tense and Register: A story might abruptly shift from past to present tense, or a formal letter might suddenly use a casual "you" (like the T/V distinction in French or German).

As Andre Martins and Gholamreza Haffari argue in Selective Attention for Context-aware Neural Machine Translation, these are not just minor stylistic errors; they are "discourse phenomena" that break the logical flow of information.

Engineering a Memory

To solve this, researchers have moved toward Context-Aware or Document-Level MT. The goal is to give the model a "memory" of what it has already translated.

Early neural approaches, such as the Hierarchical Attention Network (HAN) proposed by Lesly Miculicich and James Henderson in Document-Level Neural Machine Translation with Hierarchical Attention Networks, used two levels of attention: one for words within the current sentence and another to "look back" at previous sentence abstractions. Similarly, Longyue Wang and Zhaopeng Tu showed that explicitly exploiting cross-sentence context could resolve lexical ambiguities that sentence-level models find impossible.

More recently, the field has shifted toward pretraining. Models like DOCmT5, developed by Aditya Siddhant and Melvin Johnson, use "structure-aware" objectives to learn how documents are reordered across languages, ensuring that the thematic flow remains intact.

The Large Language Model Era

The rise of Large Language Models (LLMs) like GPT-4 has changed the landscape. In Is ChatGPT A Good Translator?, Wenxiang Jiao and colleagues found that while specialized models are fast, LLMs excel at utilizing long-range context.

However, LLMs introduce new problems. Yachao Li and Min Zhang identified a "coverage issue" in Enhancing Document-level Translation via Translation Mixed-instructions: when translating very long documents, LLMs often "lose the plot," omitting sentences or hallucinating content toward the end of the text.

To combat this, systems like DelTA, introduced by Yutong Wang and Fandong Meng, use multi-level memory structures. Instead of just cramming the whole document into a context window, they track proper nouns and bilingual summaries separately, mimicking how a human translator might keep a "glossary" and "outline" by their side.

Measuring What Matters

If the problem is global coherence, then our metrics must be global too. Standard metrics like BLEU are "blind" to discourse; they only check if small groups of words match a reference.

To address this, Yuchen Eleanor Jiang proposed BlonD, a metric that specifically rewards models for maintaining tense consistency and correct pronoun usage across sentences. Similarly, the DiP Benchmark by Prathyusha Jwalapuram and Shafiq Joty provides a "stress test" for discourse phenomena, revealing that even sophisticated models often fail to outperform simple baselines on these critical linguistic links.

The research direction is clear: moving from "translating sentences" to "reconstructing narratives." Whether through hierarchical attention, structured memory, or better instruction tuning, the goal is to ensure that a translated document reads not like a collection of accurate fragments, but like a single, cohesive thought.

Papers in this version

Showing 60 of 335 papers.

History & citationVersion 1 · 2026-10-04

Split into other directions · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_document_level_machine_translation_and_d_017c77a2,
  title = {Document Level Machine Translation and Discourse Coherence},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/document-level-machine-translation-and-discourse-coherence/rdr_017c77a2ac0c475b88156652e1c34b7b},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_document_level_machine_translation_and_d_017c77a2_v1,
  title = {Document Level Machine Translation and Discourse Coherence},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/document-level-machine-translation-and-discourse-coherence/rdr_017c77a2ac0c475b88156652e1c34b7b/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}