Abstract
Modern sentence-level NMT systems often produce plausible translations of isolated sentences. However, when put in context, these translations may end up being inconsistent with each other. We propose a monolingual DocRepair model to correct inconsistencies between sentence-level translations. DocRepair performs automatic post-editing on a sequence of sentence-level translations, refining translations of sentences in context of each other. For training, the DocRepair model requires only monolingual document-level data in the target language. It is trained as a monolingual sequence-to-sequence model that maps inconsistent groups of sentences into consistent ones. The consistent groups come from the original training data; the inconsistent groups are obtained by sampling round-trip translations for each isolated sentence. We show that this approach successfully imitates inconsistencies we aim to fix: using contrastive evaluation, we show large improvements in the translation of several contextual phenomena in an English-Russian translation task, as well as improvements in the BLEU score. We also conduct a human evaluation and show a strong preference of the annotators to corrected translations over the baseline ones. Moreover, we analyze which discourse phenomena are hard to capture using monolingual data only.
Introduction
Standard Neural Machine Translation (NMT) systems typically operate on one sentence at a time. While they produce high-quality individual sentences, they often fail at the document level. For example, a model might translate "the cat" using a masculine pronoun in one sentence and a feminine one in the next, or switch between formal and informal "you" mid-conversation. The primary challenge in fixing this is data scarcity. Most "context-aware" NMT models require document-level parallel data (source and target documents paired together), which is significantly rarer than sentence-level pairs. This research investigates whether contextual inconsistencies can be corrected using only the massive amounts of monolingual text available in the target language.
Methodology: The DocRepair Model
The proposed DocRepair is a standalone Transformer-based sequence-to-sequence model that acts as a post-processor for the output of a standard NMT system. The core innovation is a training strategy that uses a "denoising" approach to simulate translation errors using only monolingual data.
The training process involves three primary steps:
- Selection: A sequence of sentences is taken from a high-quality monolingual document in the target language (e.g., Russian).
- Corruption: "Round-trip translation" is applied to each sentence individually (e.g., Russian → English → Russian). Since this is done sentence-by-sentence, context is lost, and the resulting sentences often contain inconsistencies like gender flips or lexical shifts.
- Repair: The DocRepair model is trained to map this "broken," inconsistent sequence back to the original, consistent monolingual sequence.
At inference time, a document is translated sentence-by-sentence with any NMT system, and the resulting sequence is passed through DocRepair to harmonize the context.
Experimental Setup
The model was trained on 30 million groups of 4 consecutive sentences from the OpenSubtitles2018 corpus. Evaluation was conducted using standard BLEU scores for general quality, specialized contrastive test sets for discourse phenomena (Deixis, Lexical Cohesion, Inflectional Ellipsis, and VP Ellipsis), and blind human side-by-side comparisons.
Results and Analysis
DocRepair achieved a BLEU score improvement of +0.7 over the baseline, outperforming CADec, a state-of-the-art context-aware model that utilizes parallel data. Significant gains were observed in discourse-level accuracy, particularly in deixis and lexical cohesion.
Key findings include:
- Monolingual Sufficiency: The internal logic of the target language is often enough to identify and fix "machine-translated-ness" without seeing the source text. Speaker gender consistency and terminology can be inferred from the surrounding target text.
- Synthetic Error Effectiveness: Round-trip translation effectively generates realistic errors, forcing the model to learn document structure recovery from fragmented inputs.
- Ellipsis Challenges: The model struggled with VP Ellipsis (omitted verbs). This is likely because round-trip translation rarely produces the specific types of errors associated with ellipsis, leaving the model untrained for these cases.
In blind human evaluations, annotators preferred the repaired version 73% of the time when DocRepair modified the baseline translation, confirming the practical utility of the approach.
Conclusion
DocRepair demonstrates that document-level consistency can be significantly improved using only monolingual target-language data. By decoupling the repair process from the base NMT system, the model remains highly portable and applicable to any existing translation pipeline. Future work will focus on extending the context window and improving performance on complex phenomena like VP ellipsis through more sophisticated synthetic data generation.
Summary updated