Introduction
Indonesian is a language spoken by over 200 million people, yet it remains relatively under-researched in the field of Neural Machine Translation (NMT). A significant challenge is the "data hunger" of NMT models, coupled with the historical scarcity of large-scale parallel corpora for Indonesian. Furthermore, "spoken language" in Indonesian is highly nuanced, with a vast linguistic gap between colloquial dialogue (e.g., movie subtitles) and formal speech (e.g., TED Talks). This gap is characterized by distinct word forms, spellings, and a complex system of pronouns that vary based on social context.
This paper, presented at ACL 2019, investigates whether the Transformer architecture combined with domain adaptation (fine-tuning) can bridge this gap. The goal is to help a model trained on informal data learn the "formality" required for professional spoken presentations.
Methodology
The researchers employed the Transformer model, departing from the Recurrent Neural Networks (RNNs) used in previous Indonesian NMT research. The core strategy involved domain adaptation via fine-tuning across five training scenarios:
- IN: Training solely on a small in-domain dataset.
- OUT: Training solely on a large out-of-domain dataset.
- OUT+DA: Pre-training on out-of-domain data followed by fine-tuning on in-domain data.
- MIX: Training on a simple mixture of both datasets.
- MIX+DA: Training on a mixture followed by fine-tuning on in-domain data.
Datasets
The study utilized two primary datasets:
- Out-of-Domain (Colloquial): OpenSubtitles2018, containing approximately 9.2 million sentence pairs representing informal, dialogue-heavy language.
- In-Domain (Formal Speech): The TEDtalk corpus, containing approximately 107,000 sentence pairs representing a higher level of formality.
Evaluation was conducted using standard TEDtalk test sets (tst2013 through tst2017).
Results and Discussion
The experimental results demonstrate that quality often trumps quantity in NMT. Interestingly, the model trained only on the small in-domain dataset (IN) outperformed the model trained on the massive out-of-domain dataset (OUT) when tested on speech data, highlighting the sensitivity of NMT to domain mismatch.
As shown in the results table, the MIX+DA approach yielded the highest performance, achieving a BLEU score of 29.10 for EN-ID translation. This suggests that while large-scale subtitle data provides a strong grammatical foundation, the final fine-tuning phase is essential for "polishing" the model to meet specific stylistic requirements.
Qualitative Analysis of Formality
Beyond quantitative gains, the domain-adapted models exhibited a remarkable ability to learn implicit sociolinguistic cues. Without explicit formality tags, the models learned to adjust registers based on the distribution of the fine-tuning data.
A key finding was the successful transition of pronouns. For instance, the model shifted from informal pronouns like aku ("I") and kau ("you") to their formal counterparts saya and Anda. In a blind human evaluation with 48 native speakers, the domain-adapted system was judged "more formal" in 70% of cases compared to the non-adapted version. The model also learned to prefer formal lexical choices, such as using dapat instead of bisa for "can."
Conclusion
This research highlights the effectiveness of the Transformer architecture and domain adaptation for English-Indonesian translation in the spoken language domain. Future work could involve hyperparameter tuning specific to the EN-ID pair and more granular analysis of lexical triggers. Additionally, a direct comparison between Transformer and RNN-based models in this low-resource context remains a valuable avenue for further study.
Summary updated