TL;DR
This paper explores Neural Machine Translation (NMT) for the under-resourced language pair of Myanmar (Burmese) and Rakhine (Arakanese), a closely related dialect. They compare three NMT architectures (RNN, Transformer, CNN) and find that Syllable-BPE segmentation yields the best results across the board, with the Transformer performing best overall for NMT.
What Problem Does This Paper Address?
Machine translation (MT) has historically struggled with language varieties and dialects, often neglecting the nuances of spelling, lexical, and grammatical divergences. This is particularly true in regions like Myanmar, which boasts numerous ethnic groups and dialectal variations within its standard language. The authors aim to bridge this gap by developing a Myanmar-Rakhine parallel corpus and applying state-of-the-art NMT techniques to this under-resourced language pair. They highlight that while standard Myanmar speakers may find dialects difficult to follow, MT systems have largely overlooked this challenge.
Methods
The core of this research involves implementing and comparing three prominent NMT architectures:
- Recurrent Neural Networks (RNNs): Specifically, they mention stacked RNNs with attention, utilizing GRU and LSTM memory cells.
- Transformer: This architecture, which relies solely on attention mechanisms, is a key focus.
- Convolutional Neural Networks (CNNs): Fully convolutional models are also explored.
A crucial aspect of their methodology is the investigation of two segmentation schemes for word embeddings:
- Word-BPE (Byte Pair Encoding): This standard subword segmentation method breaks words into common character sequences.
- Syllable-BPE: This novel approach segments words into their constituent syllables, which the authors hypothesize might be more effective for morphologically rich languages like Myanmar.
They also compare their NMT results against a Statistical Machine Translation (SMT) system (specifically, the OSM approach) to benchmark performance.
Datasets
The primary dataset used in this study is a newly developed Myanmar-Rakhine parallel text corpus. This corpus was created by translating sentences from the ASEAN-MT Parallel Corpus (which is in the travel domain) into Rakhine by native speakers. The translated corpus was then checked by a Rakhine newspaper editor.
For training, development, and evaluation, they utilized:
- 14,076 sentences for training
- 2,485 sentences for development
- 1,812 sentences for evaluation
Key Insights and Ideas
- Syllable-BPE is King (for this pair): The most significant finding is that Syllable-BPE segmentation consistently outperforms Word-BPE for both SMT and NMT systems when translating between Myanmar and Rakhine. This suggests that capturing syllable-level units is crucial for handling the linguistic characteristics of this language pair.
- Transformer Dominance (with caveats): While the Transformer architecture generally achieves the highest NMT scores when using Word-BPE, the CNN with Syllable-BPE segmentation surprisingly outperforms both the RNN and Transformer in some configurations. This highlights the interplay between architecture and segmentation.
- Dialectal Translation is Achievable: The study demonstrates that high-quality NMT is feasible even for closely related dialects, which have often been overlooked in MT research. This opens doors for improving communication across linguistic variations.
- Rakhine-Myanmar vs. Myanmar-Rakhine: The paper notes that while CNN achieved the top BLEU score for Myanmar-Rakhine, the OSM SMT system performed best for Rakhine-Myanmar translation. This suggests potential asymmetries in translation performance between the two directions.
Key Results
- Syllable-BPE Superiority: Syllable-BPE segmentation consistently yielded higher BLEU scores than Word-BPE for both SMT and NMT systems. For instance, in Table 5, Syllable-BPE achieved scores in the low 80s for NMT, while Word-BPE scores were in the high 60s to low 70s.
- Top NMT Performance: The Transformer architecture with Word-BPE segmentation achieved the highest BLEU scores for both Myanmar-Rakhine and Rakhine-Myanmar translations among the NMT models.
- CNN's Syllable-BPE Strength: CNN with Syllable-BPE segmentation achieved a higher score than RNN and Transformer in some configurations, indicating its strong performance with this segmentation.
- SMT vs. NMT: Table 5 shows that for Rakhine-Myanmar translation, the OSM SMT system achieved the best BLEU score (84.36), slightly outperforming the best NMT result (83.75 from CNN). However, for Myanmar-Rakhine, CNN achieved the top score.
Limitations and Future Work
- Limited Scope of Segmentation: The authors acknowledge that they only investigated two subword segmentation schemes (Word-BPE and Syllable-BPE). They plan to explore SentencePiece as another subword segmentation scheme for future work.
- Focus on Specific Architectures: While they compared RNN, Transformer, and CNN, other NMT architectures might yield different results.
- Future Language Pairs: The researchers intend to extend their work to other Myanmar dialect languages, such as Myeik and Dawei, to further investigate NMT for these under-resourced varieties.
- Domain Specificity: The corpus was derived from the travel domain. Future work could explore performance on different domains.
Summary updated