Neural Machine Translation into Language Varieties

Surafel Melaku Lakew, Aliia Erofeeva, Marcello FedericoEMNLP 2018

TL;DR

This paper tackles the challenge of neural machine translation (NMT) for language varieties, which are often overlooked by current systems. The authors propose and evaluate multilingual NMT approaches that leverage shared representations to improve translation quality for similar languages, demonstrating significant gains over variety-specific models, especially in low-resource settings.

What Problem Does This Paper Address?

Machine translation (MT) has made incredible strides, but it often struggles with the nuances of language varieties. While major online MT services handle well-resourced languages, they tend to treat similar language varieties (like Brazilian vs. European Portuguese, or Canadian vs. European French) as a single entity. This leads to inconsistent translations that mix lexical items and grammatical structures specific to different varieties. The paper highlights that even standardized national varieties, let alone dialects, are largely neglected. This is problematic because the goal of MT is to facilitate global communication, which should extend to understanding and generating text that respects these linguistic differences. The challenge is compounded by the fact that parallel data is often not labeled at the variety level, and training data can be highly unbalanced, making it difficult to train effective NMT models for less-resourced varieties.

Methods

The core of this work lies in exploring multilingual NMT strategies to handle language varieties. The authors build upon the idea of using a single NMT model for multiple target languages, a concept known as multilingual NMT. Their approach involves prepending a special language variety token to the input string, allowing a single encoder-decoder architecture to handle different target varieties.

They investigate three main scenarios for training NMT models:

  • Supervised: Using fully labeled parallel data for each specific language variety.
  • Unsupervised: Using only unlabeled parallel data, where the model must infer the variety.
  • Semi-supervised: A combination of labeled and unlabeled data, where unlabeled data is either used as is or automatically labeled.

A key innovation is their exploration of semi-supervised multilingual NMT. They propose models that leverage unlabeled data by either:

  1. M-U: Using unlabeled data without any specific variety label.
  2. M-C2: Using a language variety identification module to automatically assign labels (A or B) to unlabeled data.
  3. M-C3: An enhancement of M-C2, which can assign one of three labels: A, B, or AU B (unlabeled).

For language variety identification, they employ a fastText model trained on word and character n-grams.

Datasets

The primary datasets used in this research are:

  • WIT³ TED Talks collection: Used for supervised and semi-supervised experiments, providing parallel text for English to various language varieties.
  • OpenSubtitles2018: Utilized to augment the English-European/Brazilian Portuguese data for large-scale experiments.

The paper does not introduce new benchmarks but rather applies existing datasets to the novel problem of NMT for language varieties.

Key Insights and Ideas

  • Multilingual NMT is key for similar languages: Training a single NMT model on multiple similar language varieties with shared representations significantly outperforms variety-specific models, especially when dealing with low-resource scenarios.
  • Unlabeled data can be leveraged effectively: Semi-supervised approaches that automatically label unlabeled data (M-C2, M-C3) generally perform better than using unlabeled data without any form of labeling (M-U).
  • Automatic labeling is beneficial, even with imperfect classifiers: Even when the language variety classifier is not perfect, automatically labeling unlabeled data leads to improvements, suggesting that exploiting all available data is crucial.
  • Variety-specific models can be outperformed by multilingual ones: In low-resource settings, the multilingual approach (Mul) often surpasses even the best supervised variety-specific models (Spec).
  • Data imbalance is a significant challenge: The unsupervised generic model (Gen), which mixes data from different varieties without distinction, can perform well on under-resourced varieties but suffers from bias when one variety dominates the training data.

Key Results

  • Low-resource setting: The multilingual NMT model (Mul) consistently outperformed unsupervised generic (Gen) and supervised variety-specific (Spec) systems across all evaluated language pairs. Semi-supervised models (M-C2, M-C3) achieved performance comparable to or better than the best supervised models.
  • High-resource setting: While variety-specific models (Spec) performed very well, the multilingual NMT model (Mul) achieved comparable results with a simpler architecture. The unsupervised generic model (Gen) performed significantly worse than supervised approaches.
  • Semi-supervised vs. Supervised: In the high-resource setting, the best semi-supervised model (M-C3) was comparable to or even better than the best supervised model (Mul), suggesting the value of leveraging unlabeled data.
  • Translation Consistency: The proposed variety-specific systems generated consistent translations for dialect-specific terms (e.g., "refrigerator"), whereas Google Translate often defaulted to one variety's translation.

Limitations and Future Work

  • Source-side varieties: The current work focuses on target-side language varieties. The authors plan to extend their approach to handle varieties in the source language as well.
  • Transfer learning: They intend to investigate the application of transfer learning techniques for language varieties, potentially by expanding their "Ada" adaptation approach.
  • Ambiguity in labeling: The paper notes that in the semi-supervised setting, the M-C3 model's "third" label (AU B) was used for only a small percentage of data, indicating potential challenges in confidently assigning labels even with three classes.
  • Bias in unsupervised models: The unsupervised generic model's performance degradation on dominant varieties highlights the challenge of data imbalance.

Summary updated

BibTeX
@inproceedings{lakew2018neural,
  author = {Surafel Melaku Lakew and Aliia Erofeeva and Marcello Federico},
  title = {{Neural Machine Translation into Language Varieties}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2018},
}
Figure