TL;DR
This paper tackles the challenging task of building an Arabic-to-Turkish machine translation system, a low-resource language pair with rich morphology. They introduce novel methods for data collection and a unique morphological abstraction technique for Turkish to improve translation quality, achieving competitive results against commercial systems.
What Problem Does This Paper Address?
Developing robust machine translation (MT) systems relies heavily on large amounts of parallel data. However, for many language pairs, especially those less commonly studied in MT research, such data is scarce. This is particularly true for Arabic-Turkish, a pair with significant linguistic differences, including highly agglutinative Turkish and morphologically complex Arabic. Existing parallel corpora often rely on English as an intermediary (pivoting), which can propagate errors. Furthermore, morphologically rich languages pose a challenge for standard MT models due to their vast vocabulary and complex word formation, leading to data sparsity and out-of-vocabulary (OOV) issues. This paper aims to address these challenges by creating a dedicated Arabic-to-Turkish MT system for the news domain, focusing on data acquisition and morphological handling.
Methods
The authors employed a multi-pronged approach to data collection and model development:
- Data Collection:
- Human Translation: Arabic news texts were translated into Turkish by professional translators.
- Web Crawling: Existing open-source Arabic-Turkish parallel texts were exploited, though often out-of-domain.
- Back-translation: Monolingual Turkish news corpora were translated into Arabic using existing MT systems to augment the parallel data.
- Morphological Abstraction for Turkish: To handle Turkish's agglutinative nature and reduce vocabulary size, they developed a novel abstraction method. This technique simplifies morpheme suffixes by normalizing variations caused by vowel harmony and consonant assimilation. This abstraction allows for easier generation of the correct surface form after translation.
- Segmentation and Abstraction Experiments: They experimented with different representations of Turkish words: surface form, segmented subwords, abstracted subwords, and a combination of both.
- Neural Machine Translation (NMT): The state-of-the-art Marian NMT toolkit was used, employing the Transformer architecture.
Datasets
- In-domain Human Translated Data: Approximately 160K Arabic sentences translated to Turkish.
- Out-of-Domain (OOD) Parallel Data:
- WIT (Web Inventory of Transcribed and Translated Talks): ~130K Arabic-Turkish sentences.
- OpenSubtitles2018: ~21M Arabic-Turkish sentences.
- Monolingual Turkish Data: CNN-Turk and Aljazeera news corpora (~3M sentences) used for back-translation.
- Development and Test Sets: 1,600 sentences for development and 1,357 sentences for testing, with multiple Turkish references.
Key Insights and Ideas
- Morphological Abstraction is Key: The paper strongly suggests that their novel morphological abstraction technique for Turkish is more effective than simple segmentation for improving translation quality, especially when combined with NMT and vocabulary fitting methods like BPE. It helps normalize variations caused by Turkish's complex phonological rules.
- Data Augmentation is Crucial: Back-translation, even with a high ratio of synthetic data to real data, significantly boosts performance, highlighting its value for low-resource language pairs.
- OOD Data Still Helps: Even though OOD data (like subtitles) is not ideal, it can still contribute to performance gains, albeit to a lesser extent than in-domain data.
- News Domain Specifics: The news domain offers advantages like publicly available parallel data and a standardized Arabic dialect (Modern Standard Arabic).
Key Results
- The baseline system achieved a BLEU score of 15.91.
- Augmenting with backtranslated data (MONO) improved BLEU by +6 points to 21.78.
- Further incorporating OOD subtitles data (OOD2) pushed the score to 24.09.
- The proposed morphological abstraction method (experiment 5) achieved a BLEU score of 24.92, outperforming segmentation alone (24.13) and combined abstraction/segmentation (24.83).
- Their best system achieved a BLEU score of 24.83, significantly outperforming Google Translate (20.70) and Yandex Translate (11.82) on their test set.
- The morphological abstraction process successfully generated correct surface forms for 98% of words using a generation tool.
Limitations and Future Work
- Arabic Segmentation: The paper acknowledges that Arabic segmentation was out of scope and relied on existing tools (MADAMIRA). Further work could explore Arabic morphology more deeply.
- OOD Data Quality: While OOD data helped, its limited overlap with the news domain meant improvements were not as substantial as expected. Future work could focus on better filtering or domain adaptation of OOD data.
- Abstraction Generalization: The authors suggest their abstraction algorithm could be applied to other European languages for Turkish translation tasks.
- Further Morphological Exploration: The paper hints at exploring more sophisticated morphological analysis and generation techniques for both languages.
Summary updated