Neural Machine Translation: Hindi-Nepali

Sahinur Rahman Laskar, Partha Pakray, Sivaji BandyopadhyayACL 2019

TL;DR

This paper explores Neural Machine Translation (NMT) for the Hindi-Nepali language pair, a relatively under-resourced area. The authors leverage existing NMT toolkits and attention mechanisms to achieve competitive results, highlighting the potential of NMT for similar language translation despite data limitations.

What Problem Does This Paper Address?

Machine Translation (MT) systems are crucial for bridging language barriers, but their effectiveness is often limited by the availability of parallel data. This is particularly true for language pairs that are linguistically similar but may not have extensive parallel corpora. The paper addresses the challenge of building a robust MT system for the Hindi-Nepali language pair, a task that is still in its early stages. Existing MT approaches, like rule-based or statistical methods, have limitations in handling linguistic complexities and require significant manual effort. NMT, while promising, still faces the hurdle of data scarcity for less common language pairs.

Methods

The authors employ Neural Machine Translation (NMT) with an attention mechanism. The core of their approach involves using a standard RNN encoder-decoder architecture, specifically leveraging Long Short-Term Memory (LSTM) units to capture long-term dependencies in the source and target sentences. The attention mechanism is a key component, allowing the decoder to focus on relevant parts of the source sentence when generating each word of the target sentence, thus mitigating information loss in longer sentences. They utilize two popular open-source NMT toolkits: OpenNMT and Marian NMT. The system is trained and tested in three configurations (Run-1, Run-2, Run-3) using both primary and contrastive system types.

Datasets

The training corpus for Hindi-Nepali translation was compiled manually through back-translation using Google Translate. Sources for this corpus included Wikipedia articles in Hindi and Nepali, as well as the Bible. Additionally, data from the WMT19 shared task was utilized.

For testing, the WMT19 organizer provided specific datasets:

  • Hindi to Nepali translation: 1,567 instances.
  • Nepali to Hindi translation: 2,000 instances.

A subset of 500 instances from the training corpus was also used for validation.

Key Insights and Ideas

  • NMT's Efficacy for Similar Languages: The study demonstrates that NMT can be effectively applied to translate between linguistically similar languages like Hindi and Nepali, even with limited parallel data.
  • Attention Mechanism's Importance: The attention mechanism proves crucial for improving translation quality, especially for longer sentences, by enabling the model to selectively focus on relevant source information.
  • Toolkits Performance: The paper suggests that Marian NMT might outperform OpenNMT in both Hindi-Nepali and Nepali-Hindi translation under contrastive configurations, though further investigation is recommended.
  • Data Augmentation Value: Manual back-translation, while labor-intensive, is shown to be a viable strategy for augmenting datasets for under-resourced language pairs.

Key Results

The NMT system achieved varying performance across different configurations and directions:

  • Primary Configuration:
    • Nepali to Hindi: BLEU score of 24.6.
  • Hindi to Nepali: BLEU score of 3.7.
  • Contrastive Configuration:
    • Hindi to Nepali: Excellent BLEU score of 53.7 (using Marian NMT).
  • Nepali to Hindi: BLEU score of 49.1 (using Marian NMT).

Qualitatively, the paper notes that while the NMT system can provide perfect predictions comparable to Google Translate for some sentences, it also exhibits instances of incorrect translations, including translations into a different language or imprecise translations for long sentences.

Limitations and Future Work

The authors acknowledge that their NMT systems still require improvement in handling cases of wrong translations and translations into different languages. They also point out that the BLEU scores for both directions are relatively stable in contrastive configurations but less so in the primary configuration.

Future work suggested includes:

  • Investigating the reasons behind Marian NMT's superior performance over OpenNMT in contrastive configurations.
  • Expanding the dataset size for Hindi-Nepali translation.
  • Exploring machine translation for other similar Indian language pairs, such as Bengali-Assamese, Telugu-Kannada, and Hindi-Punjabi, to address the parallel data limitation.

Summary updated

BibTeX
@inproceedings{laskar2019neural,
  author = {Sahinur Rahman Laskar and Partha Pakray and Sivaji Bandyopadhyay},
  title = {{Neural Machine Translation: Hindi-Nepali}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2019},
}
Figure