Polygon-Net: A General Framework for Jointly Boosting Multiple Unsupervised Neural Machine Translation Models

Chang Xu, Tao Qin, Gang Wang, Tie-Yan Liu

Abstract

Unsupervised Neural Machine Translation (UNMT) has emerged as a powerful paradigm for low-resource language pairs. However, existing bilingual UNMT methods often suffer from a vast hypothesis space, leading to poor translation quality. This paper introduces Polygon-Net, a general framework that jointly boosts multiple UNMT models by leveraging auxiliary languages. By enforcing semantic consistency across different translation paths, Polygon-Net introduces stronger training signals. Experimental results on UN and WMT datasets demonstrate that Polygon-Net consistently outperforms bilingual baselines, with performance scaling as more languages are integrated.

Introduction

Standard Neural Machine Translation (NMT) requires massive amounts of parallel data, which is often unavailable for many language pairs. Unsupervised NMT (UNMT) addresses this by utilizing only monolingual corpora, typically through back-translation between two languages. However, this bilingual approach often fails to sufficiently constrain the model, leaving a large "hypothesis space" where models can satisfy training objectives without producing accurate translations. Polygon-Net addresses this by adding more constraints through a multi-language "polygon" structure.

Constraining the Hypothesis Space

The core intuition behind Polygon-Net is that adding auxiliary languages restricts the search space for the translation model. In a bilingual setup, the model has significant freedom to deviate from the optimal translation while still satisfying back-translation objectives. By introducing a third language, the model must now be consistent across multiple paths, acting as a powerful regularizer that forces the models toward the "golden" translation model.

Comparison of hypothesis spaces in bilingual (a) vs. trilingual (b) UNMT.

Trilingual Semantic Consistency

Polygon-Net treats languages as nodes in a polygon and translation models as directed edges. The framework enforces semantic consistency across different routes. For any three languages ($X, Y, Z$), the direct translation $Z \to Y$ should yield a result semantically consistent with the indirect path $Z \to X \to Y$.

The trilingual setup in Polygon-Net.

To implement this, the authors designed a trilingual loss function that minimizes the disagreement between direct and indirect translation pipelines. Since calculating the probability of an indirect path involves summing over all possible intermediate sentences, the authors employ importance sampling to estimate this loss efficiently by focusing on the most relevant sentences.

Training and Scaling

Polygon-Net utilizes a "round-robin" training strategy where all models in the system are updated iteratively, allowing them to mutually boost each other's performance. While multi-hop paths can become complex as more languages are added, the authors prove that optimizing for all possible language triplets is sufficient to optimize the entire polygon. This keeps computational complexity manageable, scaling linearly rather than exponentially.

Experimental Results

The researchers evaluated Polygon-Net on the UN Corpus and WMT (News Crawl) benchmarks across 12 language pairs. Key findings include:

  • Consistent Outperformance: Polygon-Net consistently surpassed strong bilingual UNMT baselines. For instance, on the WMT'13 English-French task, the bilingual baseline achieved 18.83 BLEU, while a 5-language Polygon-Net reached 20.27 BLEU.
  • The "More is Better" Effect: Translation quality improved monotonically as more auxiliary languages were added to the framework.
  • Universal Semantic Grounding: Adding "non-mainstream" languages like Czech improved the performance of "mainstream" pairs like English-French, suggesting that auxiliary languages provide a form of universal semantic grounding.
  • Gains in Difficult Pairs: Significant improvements were noted in challenging pairs; for German-French, BLEU scores jumped from 4.06 to 9.45.

Conclusion

Polygon-Net provides a robust framework for improving unsupervised translation by leveraging the collective power of multiple languages. By transforming the UNMT problem from a bilingual task into a multi-language consistency problem, it effectively "tightens" the search space and produces higher-quality translations. Future work may explore more complex graph topologies and automated selection of the most beneficial auxiliary languages for specific target pairs.

Summary updated

Abstract

Neural machine translation (NMT) has achieved great success. However, collecting large-scale parallel data for training is costly and laborious. Recently, unsupervised neural machine translation has attracted more and more attention, due to its demand for monolingual corpus only, which is common and easy to obtain, and its great potentials for the low-resource or even zero-resource machine translation. In this work, we propose a general framework called Polygon-Net, which leverages multi auxiliary languages for jointly boosting unsupervised neural machine translation models. Specifically, we design a novel loss function for multi-language unsupervised neural machine translation. In addition, different from the literature that just updating one or two models individually, Polygon-Net enables multiple unsupervised models in the framework to update in turn and enhance each other for the first time. In this way, multiple unsupervised translation models are associated with each other for training to achieve better performance. Experiments on the benchmark datasets including UN Corpus and WMT show that our approach significantly improves over the two-language based methods, and achieves better performance with more languages introduced to the framework.
BibTeX
@inproceedings{xu2019polygon,
  author = {Chang Xu and Tao Qin and Gang Wang and Tie-Yan Liu},
  title = {{Polygon-Net: A General Framework for Jointly Boosting Multiple Unsupervised Neural Machine Translation Models}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2019},
  doi = {10.24963/ijcai.2019/739},
}
Figure