Phrase-level System Combination for Machine Translation Based on Target-to-Target Decoding

Wei-Yun Ma, Kathleen R. McKeownAMTA 2020 · OpenReview

Abstract

Machine translation (MT) system combination aims to produce a single high-quality translation by merging outputs from multiple MT engines. Traditional word-level approaches, such as Confusion Network decoding, often compromise local coherence. This paper proposes Target-to-Target Decoding (TTD), a framework that treats system combination as a translation task. By extracting monolingual paraphrases from alignments between a selected backbone and other hypotheses, TTD utilizes a phrase-based decoder to generate superior translations. Evaluations on CE-NIST and GE-WMT benchmarks show that TTD significantly improves BLEU scores over individual systems and ranks among the top-performing combination methods.

Introduction

System combination is a critical technique for improving MT quality by leveraging the complementary strengths of different translation models. The standard approach, word-level Confusion Network (CN) decoding, frequently results in ungrammatical "Frankenstein" sentences due to the fragmentation of natural phrases. While phrase-level combination has been explored, existing methods often rely on complex heuristics or rigid structures. This work introduces Target-to-Target Decoding (TTD), which simplifies the process by reformulating combination as a standard phrase-based translation task.

The Target-to-Target Decoding Framework

The TTD framework operates in four primary stages:

  1. Backbone Selection: A single hypothesis is chosen as the structural template (backbone) using a log-linear model that incorporates system weights and language model scores.
  2. Monolingual Alignment: The backbone is aligned with other system hypotheses using the Translation Error Rate Plus (TERp) metric, which accounts for shifts, synonyms, and stems.
  3. Paraphrase Extraction: Following the principles of phrase-based MT, monolingual phrase pairs (paraphrases) are extracted from these alignments. This process allows for many-to-many mappings, creating a rich repository of translation options.
  4. Log-Linear Combination and Decoding: A log-linear model scores potential translations based on TERp-based confidence, lexical weighting, and syntactic indicators. A key innovation is the use of System-Specific Language Models (LMs) to estimate N-gram consensus. Finally, the Moses decoder is used to "translate" the backbone into the optimal combined output.
Monolingual alignment between a backbone sentence (Eb) and a hypothesis (Eh) using TERp.

Experimental Setup and Datasets

The TTD system was evaluated on two major benchmarks:

  • CE-NIST: Chinese-to-English translations from the NIST 2008 Open MT evaluation (newswire and web data).
  • GE-WMT: German-to-English translations from the WMT 2011 system combination shared task.

Results and Analysis

The experimental results demonstrate the effectiveness of the TTD approach. On the CE-NIST dataset, the full TTD model (P+cs+sl+lex+syn) achieved a BLEU score of 32.85, significantly surpassing the best individual system (30.16) and the backbone (30.89).

Ablation study on the CE-NIST dataset comparing word-level (W) and phrase-level (P) combination features.

Ablation studies, as shown in the table above, highlight that phrase-level combination (P) consistently outperforms word-level combination (W). Furthermore, the inclusion of system-specific LMs and syntactic features provided incremental gains. Interestingly, the authors found that adding a reordering model did not significantly improve performance, suggesting that the selected backbones were already structurally sound. In the WMT 2011 competition, the TTD model ranked in the top two among ten participating systems, confirming its competitiveness.

Conclusion

Target-to-Target Decoding offers a robust and effective framework for MT system combination. By leveraging the mature infrastructure of phrase-based MT, TTD avoids the pitfalls of word-level merging and achieves state-of-the-art results. Future work will focus on improving backbone selection and exploring incremental alignment techniques to further enhance translation quality.

Summary updated

BibTeX
@inproceedings{ma2020phrase,
  author = {Wei-Yun Ma and Kathleen R. McKeown},
  title = {{Phrase-level System Combination for Machine Translation Based on Target-to-Target Decoding}},
  booktitle = {AMTA},
  year = {2020},
  url = {https://openreview.net/forum?id=EljvHtL16M},
}
Figure