NICT’s Machine Translation Systems for the WMT19 Similar Language Translation Task

Benjamin Marie, Raj Dabre, Atsushi FujitaACL 2019

TL;DR

NICT's team participated in the WMT19 Similar Language Translation Task for Spanish-Portuguese, achieving strong results by combining state-of-the-art neural and statistical machine translation systems. Their approach leveraged extensive back-translation to augment training data and a novel reranking framework, demonstrating the power of hybrid systems even for closely related languages.

What Problem Does This Paper Address?

The paper addresses the challenge of building high-quality machine translation (MT) systems, specifically for language pairs that are closely related, like Spanish and Portuguese. While neural machine translation (NMT) has become dominant, the authors explore how it compares to and can be combined with traditional statistical machine translation (SMT) for such languages. They also investigate the effectiveness of unsupervised MT (USMT) when large amounts of monolingual data are available, questioning the necessity of massive parallel corpora for closely related languages.

Methods

The NICT team employed a multi-pronged approach:

  • Neural Machine Translation (NMT): They utilized the Transformer architecture, implemented in the Marian framework. To boost performance, they significantly augmented the provided parallel data with synthetic sentences generated through back-translation from monolingual corpora. They also used an ensemble of four independently trained NMT models for decoding.
  • Statistical Machine Translation (SMT): Standard SMT systems were built using the Moses toolkit, incorporating word alignments, phrase tables, and lexicalized reordering models. A 4-gram language model was trained on monolingual data.
  • Unsupervised SMT (USMT): Recognizing the potential for closely related languages, they built an USMT system. This involved inducing phrase tables from monolingual data using bilingual word embeddings and then tuning a Moses system. They refined this system through multiple steps of synthetic data generation.
  • Combination and Reranking: The core of their submission was a combination of NMT and SMT outputs. They generated n-best lists from both NMT and SMT systems and then used a reranking framework. This framework employed features from the individual NMT models, an SMT language model, and sentence-level translation probabilities to select the best hypothesis from the merged lists.

Datasets

The primary datasets used were those provided by the WMT19 organizers for the Spanish-Portuguese task:

  • Parallel Data: Used for training supervised NMT and SMT systems.
  • Monolingual Data: Primarily "News Crawl" corpora, used for:
  • Training unsupervised MT systems.
  • Generating synthetic parallel data via back-translation to augment supervised training.
  • Training SMT language models.
  • Development Data: Used for tuning MT systems and training the reranking framework.

No new benchmarks were introduced; the focus was on applying existing techniques to the provided WMT19 datasets.

Key Insights and Ideas

  • Hybrid Systems Shine: Combining NMT and SMT through reranking significantly improved translation quality, even when SMT alone might underperform NMT. This suggests that NMT and SMT can be complementary.
  • Back-Translation is Powerful: Augmenting parallel data with synthetically generated sentences via back-translation from large monolingual corpora is a highly effective strategy for improving NMT performance.
  • USMT's Potential for Close Languages: For languages as similar as Spanish and Portuguese, unsupervised MT can achieve surprisingly good results, coming close to supervised systems. This challenges the notion that massive parallel data is always a prerequisite for high-quality MT.
  • Reranking Adds Value: A sophisticated reranking mechanism that leverages features from both NMT and SMT can effectively select better hypotheses from combined n-best lists than the top-ranked hypothesis from individual systems.

Key Results

  • NICT's primary submission, a combination of SMT and NMT, achieved the second and third ranks for Portuguese-to-Spanish and Spanish-to-Portuguese translation directions, respectively, based on BLEU scores.
  • The reranking approach provided a substantial improvement, for example, 1.6 BLEU points for es→pt on the development data.
  • The unsupervised SMT (USMT) system performed remarkably well, scoring only 5.4 BLEU points below the primary model for es→pt on the test data.

Limitations and Future Work

  • Test Set BLEU: The authors note that due to the test data references not being released at the time of writing, they could not compute BLEU scores for all configurations on the test set, using "-" for those cases.
  • USMT vs. UNMT: While they chose USMT over unsupervised NMT (UNMT) based on prior work suggesting USMT's slight outperformance for this task, further direct comparisons could be explored.
  • Future Work: The paper implicitly suggests that for closely related languages, further exploration of unsupervised or semi-supervised methods could be fruitful, potentially reducing the reliance on large parallel corpora. The success of the hybrid approach also points to continued research in effective combination and reranking strategies.

Summary updated

BibTeX
@inproceedings{marie2019nicts,
  author = {Benjamin Marie and Raj Dabre and Atsushi Fujita},
  title = {{NICT’s Machine Translation Systems for the WMT19 Similar Language Translation Task}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2019},
}
Figure