TL;DR
This paper presents the IITP-MT system for the Gujarati-English news translation task at WMT 2019. Facing the challenge of Gujarati being a low-resource language with limited parallel data, the authors successfully leveraged back-translation to augment their training data, significantly improving the performance of their Transformer-based neural machine translation (NMT) systems.
What Problem Does This Paper Address?
Machine translation (MT) has seen remarkable progress with Neural Machine Translation (NMT) systems, particularly those based on the Transformer architecture. However, NMT models are notoriously data-hungry, requiring large amounts of high-quality parallel text to achieve state-of-the-art performance. This becomes a significant hurdle for low-resource languages, where such data is scarce. The Gujarati-English language pair, introduced for the first time in the WMT 2019 shared task, exemplifies this challenge. With only a few thousand parallel sentences available, training effective NMT and even traditional Statistical Machine Translation (SMT) systems is difficult. Furthermore, Gujarati and English have distinct linguistic properties, including word order (SOV vs. SVO), making the translation task inherently complex.
Methods
The IITP-MT system employs a Transformer-based NMT architecture, which is the current standard for many high-resource language pairs due to its parallelizability and effectiveness. To address the low-resource nature of Gujarati-English, the authors implemented a back-translation strategy.
- Baseline Systems: They first trained baseline phrase-based SMT (PBSMT) and Transformer-based NMT systems using only the available parallel data.
- Synthetic Data Generation: For back-translation, they trained PBSMT systems in the reverse direction (English to Gujarati and Gujarati to English). These PBSMT systems were then used to translate monolingual data (English News Crawl and Gujarati Wikipedia dumps) into synthetic parallel corpora.
- Augmented Training: The generated synthetic parallel data was then combined with the original parallel data. This augmented dataset was used to train the final Transformer-based NMT systems.
- Subword Units: All systems were trained using subword units (Byte Pair Encoding or BPE) to handle rare words and morphology effectively.
Datasets
The paper utilized the following datasets for training and tuning:
- Parallel Data:
- Bible corpus: 7,807 sentences
- govin-clean.gu-en.tsv: 10,650 sentences
- opus.gu-en.tsv: 107,637 sentences
- wikipedia.gu-en.tsv: 18,033 sentences
- wikititles-v1.gu-en.tsv: 11,671 sentences
- Total Parallel Data: 155,798 sentences
- Monolingual Data:
- Gujarati (Wikipedia dump): 382,881 sentences
- English (News crawl): 1,000,000 sentences
- Tuning Data: newsdev2019 (1,998 parallel sentences)
Key Insights and Ideas
- Back-translation is Crucial for Low-Resource Languages: The paper strongly demonstrates that back-translation is a highly effective technique for improving NMT performance when parallel data is scarce. The synthetic data generated from monolingual corpora significantly boosted the BLEU scores.
- Transformer Architecture's Potential: Even with limited parallel data, the Transformer architecture shows promise, outperforming baseline SMT systems when augmented with synthetic data.
- Gujarati's Unique Challenges: The introduction of Gujarati highlights the need for specialized techniques for low-resource languages with distinct linguistic features. The significant difference in word order between Gujarati (SOV) and English (SVO) is a key factor in translation difficulty.
Key Results
The primary quantitative results are presented in terms of BLEU scores:
- English to Gujarati:
- Baseline Transformer (original data only): 4.0 BLEU
- Transformer + Synthetic Data: 8.1 BLEU
- Gujarati to English:
- Baseline Transformer (original data only): 5.5 BLEU
- Transformer + Synthetic Data: 10.4 BLEU
These results show substantial improvements, with the augmented systems achieving significantly higher BLEU scores than their baseline counterparts. The paper also notes that the augmented NMT systems outperformed baseline SMT systems.
Limitations and Future Work
- Data Scarcity: The fundamental limitation remains the scarcity of high-quality parallel data for Gujarati. Even with back-translation, the performance is still lower compared to high-resource language pairs.
- Short Sentences: The paper mentions that many sentences in the training set are very short (2-3 tokens), which can negatively impact translation quality.
- Future Work: The authors suggest that further research into low-resource language translation, particularly for languages like Gujarati, is warranted. Exploring more advanced data augmentation techniques or unsupervised MT methods could be potential future directions.
Summary updated