Abstract
Unsupervised Neural Machine Translation (UNMT) has emerged as a powerful paradigm for low-resource language pairs. However, existing bilingual UNMT methods often suffer from a vast hypothesis space, leading to poor translation quality. This paper introduces Polygon-Net, a general framework that jointly boosts multiple UNMT models by leveraging auxiliary languages. By enforcing semantic consistency across different translation paths, Polygon-Net introduces stronger training signals. Experimental results on UN and WMT datasets demonstrate that Polygon-Net consistently outperforms bilingual baselines, with performance scaling as more languages are integrated.
Introduction
Standard Neural Machine Translation (NMT) requires massive amounts of parallel data, which is often unavailable for many language pairs. Unsupervised NMT (UNMT) addresses this by utilizing only monolingual corpora, typically through back-translation between two languages. However, this bilingual approach often fails to sufficiently constrain the model, leaving a large "hypothesis space" where models can satisfy training objectives without producing accurate translations. Polygon-Net addresses this by adding more constraints through a multi-language "polygon" structure.
Constraining the Hypothesis Space
The core intuition behind Polygon-Net is that adding auxiliary languages restricts the search space for the translation model. In a bilingual setup, the model has significant freedom to deviate from the optimal translation while still satisfying back-translation objectives. By introducing a third language, the model must now be consistent across multiple paths, acting as a powerful regularizer that forces the models toward the "golden" translation model.
Trilingual Semantic Consistency
Polygon-Net treats languages as nodes in a polygon and translation models as directed edges. The framework enforces semantic consistency across different routes. For any three languages ($X, Y, Z$), the direct translation $Z \to Y$ should yield a result semantically consistent with the indirect path $Z \to X \to Y$.
To implement this, the authors designed a trilingual loss function that minimizes the disagreement between direct and indirect translation pipelines. Since calculating the probability of an indirect path involves summing over all possible intermediate sentences, the authors employ importance sampling to estimate this loss efficiently by focusing on the most relevant sentences.
Training and Scaling
Polygon-Net utilizes a "round-robin" training strategy where all models in the system are updated iteratively, allowing them to mutually boost each other's performance. While multi-hop paths can become complex as more languages are added, the authors prove that optimizing for all possible language triplets is sufficient to optimize the entire polygon. This keeps computational complexity manageable, scaling linearly rather than exponentially.
Experimental Results
The researchers evaluated Polygon-Net on the UN Corpus and WMT (News Crawl) benchmarks across 12 language pairs. Key findings include:
- Consistent Outperformance: Polygon-Net consistently surpassed strong bilingual UNMT baselines. For instance, on the WMT'13 English-French task, the bilingual baseline achieved 18.83 BLEU, while a 5-language Polygon-Net reached 20.27 BLEU.
- The "More is Better" Effect: Translation quality improved monotonically as more auxiliary languages were added to the framework.
- Universal Semantic Grounding: Adding "non-mainstream" languages like Czech improved the performance of "mainstream" pairs like English-French, suggesting that auxiliary languages provide a form of universal semantic grounding.
- Gains in Difficult Pairs: Significant improvements were noted in challenging pairs; for German-French, BLEU scores jumped from 4.06 to 9.45.
Conclusion
Polygon-Net provides a robust framework for improving unsupervised translation by leveraging the collective power of multiple languages. By transforming the UNMT problem from a bilingual task into a multi-language consistency problem, it effectively "tightens" the search space and produces higher-quality translations. Future work may explore more complex graph topologies and automated selection of the most beneficial auxiliary languages for specific target pairs.
Summary updated