Abstract
This study investigates the impact of back-translation (BT) on machine translation (MT) for the low-resource Chinese-Vietnamese language pair. By comparing Statistical Machine Translation (SMT) and Neural Machine Translation (NMT) across different tokenization strategies and data scales, the researchers find that while BT generally enhances performance, its effectiveness varies significantly by translation direction and model type. Notably, character-based models consistently outperform word-based ones, and increasing synthetic data volume often leads to diminishing returns, challenging common assumptions in high-resource MT research.
Introduction
Neural Machine Translation (NMT) has revolutionized the field but remains heavily dependent on massive parallel datasets. For "low-resource" language pairs like Chinese and Vietnamese, such data is scarce. Back-translation—a technique where a model translates monolingual target-side data back into the source language to create "synthetic" parallel data—is a standard solution.
However, most BT research focuses on European languages. Chinese and Vietnamese present unique challenges: they belong to different language families (Sino-Tibetan vs. Austroasiatic) and use different scripts, yet they share deep historical ties and similar analytic (SVO) structures. This paper systematically tests if Western-centric BT assumptions hold true for this specific Asian language pair.
Methodology
The researchers conducted a comprehensive comparison across several dimensions:
- Model Architectures: Traditional Statistical Machine Translation (SMT) using the Moses toolkit was compared against NMT using a 2-layer LSTM (OpenNMT).
- Tokenization: Both character-based and word-based settings were tested. This is critical for Vietnamese, where spaces separate syllables rather than words, and Chinese, which uses no spaces.
- Back-Translation Pipeline: The process involved training an intermediate system (e.g., Vietnamese → Chinese) to translate monolingual data into synthetic pairs, which were then combined with authentic data to train the final system.
- Incremental Scaling: Synthetic data was added in 10k increments (from 10k to 50k) to a base of 50k authentic pairs to identify the optimal data volume.
Datasets
The study utilized a high-quality, manually processed dataset of 56,610 authentic sentence pairs from Vietnamese news websites. A separate 200k parallel corpus from various domains (textbooks, dialogues, etc.) provided the monolingual data for the back-translation process. Performance was evaluated using BLEU and METEOR metrics.
Key Insights and Results
Character-Based vs. Word-Based Modeling
Character-based models almost always outperformed word-based ones. Character-level modeling likely bypasses errors introduced by imperfect word segmentation tools and better handles the monosyllabic nature of both languages.
The "More is Better" Myth
Unlike high-resource scenarios, adding more synthetic data sometimes decreased performance. In many NMT experiments, scores peaked before reaching the maximum amount of synthetic data, suggesting that too much "noisy" synthetic data can overwhelm a small authentic dataset.
Directional Asymmetry and Model Performance
BT was significantly more effective for the Chinese → Vietnamese direction than the reverse. In the character-based Chinese → Vietnamese task, the baseline BLEU of 14.29 rose to 14.95 with 50k synthetic sentences. For NMT, BLEU rose from 11.68 to 12.34, peaking at 30k synthetic sentences.
Qualitatively, NMT proved superior at handling entity names and complex syntactic structures, such as the Chinese "把" (BA) construction, compared to SMT, which often produced less coherent translations or left names untranslated.
Conclusion
The study highlights that BT is a viable but sensitive technique for Chinese-Vietnamese MT. The results were limited by a relatively small dataset, which precluded the use of the Transformer architecture or Byte Pair Encoding (BPE). Future work will focus on expanding dataset sizes to enable Transformer-based experiments and exploring the combination of back-translation with transfer learning to further boost performance in low-resource settings.
Summary updated