Neural Machine Translation for English-Tamil

Himanshu Choudhary, Aditya Kumar Pathak, Rajiv Ratan Saha, Ponnurangam KumaraguruWMT (shared Task) 2018 · DOI

TL;DR

This paper tackles the challenge of English-to-Tamil machine translation, a difficult task due to linguistic differences. The authors propose a novel Neural Machine Translation (NMT) approach that combines pre-trained word embeddings with Byte-Pair Encoding (BPE) to significantly improve translation accuracy and address the Out-Of-Vocabulary (OOV) problem, outperforming Google Translate.

What Problem Does This Paper Address?

Translating content from English to local languages is crucial for knowledge sharing, especially in countries like India with a large population that doesn't speak English fluently. Manual translation is costly and time-consuming, making machine translation (MT) a necessity. However, translating between English and Indian languages like Tamil presents significant challenges. These include the scarcity of parallel corpora (large datasets of aligned sentences in both languages) and substantial linguistic differences. Tamil, for instance, has a Subject-Object-Verb (SOV) word order, contrasting with English's Subject-Verb-Object (SVO), and is morphologically rich and agglutinative, leading to data sparsity issues. Existing MT systems often struggle with these complexities.

Methods

The core of this paper is the development of the MIDAS translator, a Neural Machine Translation (NMT) system. The authors leverage two key techniques:

  • Word Embeddings: They utilize pre-trained word embeddings (specifically FastText) to represent English and Tamil words as dense vectors in a 300-dimensional space. This allows words with similar meanings to have similar vector representations, capturing semantic relationships.
  • Byte-Pair Encoding (BPE): BPE is employed as a data compression technique for word segmentation. It works by iteratively merging the most frequent pairs of characters or sub-word units. This helps in handling rare words and morphological variations by breaking them down into smaller, known units, thereby mitigating the Out-Of-Vocabulary (OOV) problem.

The NMT architecture itself is based on a sequence-to-sequence model with an attention mechanism. The encoder processes the source sentence (English) into a context vector, and the decoder generates the target sentence (Tamil) word by word, paying attention to relevant parts of the source sentence at each step. The authors experimented with different configurations of LSTM layers, optimizers (SGD, Adam), and attention mechanisms (Luong, Bahdanau) to find the optimal setup.

Datasets

The authors used datasets from two sources:

  • EnTamv2.0: A parallel corpus for English-Tamil translation.
  • Opus: Another collection of parallel corpora.

They combined these datasets and performed extensive preprocessing to create their final dataset. After cleaning and deduplication, they obtained 1,83,451 training sentences, 1,000 validation sentences, and 2,000 test sentences for the English-Tamil pair. A significant part of their work involved addressing data quality issues, such as removing duplicate sentences and handling tokenization challenges specific to Tamil.

Key Insights and Ideas

  • BPE is a Game-Changer for Morphologically Rich Languages: The paper strongly suggests that BPE is a highly effective method for handling the morphological complexity of languages like Tamil. By breaking down words into sub-word units, it significantly reduces the OOV problem and improves translation quality without requiring overly complex models.
  • Data Preprocessing is Crucial: The authors emphasize that meticulous data cleaning and preprocessing are not just beneficial but essential for achieving good results in NMT, especially for low-resource language pairs. Removing duplicate sentences and handling language-specific tokenization issues directly impacts model performance.
  • Word Embeddings Enhance Semantic Understanding: The use of pre-trained word embeddings allows the NMT model to better grasp the semantic nuances of words, leading to more accurate and contextually relevant translations.
  • NMT Outperforms Traditional MT: The results clearly demonstrate that their NMT approach, enhanced with word embeddings and BPE, significantly outperforms traditional machine translation techniques, including Google Translate, for the English-Tamil pair.

Key Results

The MIDAS translator achieved a BLEU score of 8.33, which is a substantial improvement over Google Translate's score of 3.75. This represents a margin of 4.58 BLEU points. The paper also reports that their model achieved comparable accuracy with a simpler architecture and less training time compared to training deep, complex neural networks. An anonymous survey of Tamil speakers indicated that translations from their MIDAS translator were preferred over Google Translate's in 71.66% of cases.

Limitations and Future Work

The authors acknowledge that while their model shows significant improvement, the BLEU scores are still relatively low, indicating room for further enhancement. They suggest that their approach can be extended to other Indian language pairs. Future work could also explore using these techniques for detecting offensive language in code-switched languages.

Summary updated

BibTeX
@inproceedings{choudhary2018neural,
  author = {Himanshu Choudhary and Aditya Kumar Pathak and Rajiv Ratan Saha and Ponnurangam Kumaraguru},
  title = {{Neural Machine Translation for English-Tamil}},
  booktitle = {WMT (shared Task)},
  year = {2018},
  doi = {10.18653/v1/w18-6459},
}
Figure