ANVITA Machine Translation System for WAT 2021 MultiIndicMT Shared Task

Pavanpankaj Vegi, Sivabhavani J, Biswajit Paul, Chitra Viswanathan, Prasanna Kumar K RACL 2021

TL;DR

This paper introduces ANVITA-1.0, a multilingual neural machine translation system designed for the WAT 2021 MultiIndicMT shared task, focusing on 20 translation directions between English and 10 Indian languages. The system leverages a Transformer architecture with shared encoders/decoders, extensive data filtering, and augmentation techniques to achieve strong performance, particularly for Indic-to-English translation.

What Problem Does This Paper Address?

Developing high-quality machine translation (MT) systems for Indian languages is a significant challenge due to their morphological richness and the common scarcity of parallel data. Many Indian languages are considered "low-resource," meaning they lack the extensive parallel corpora needed to train robust MT models. The WAT 2021 MultiIndicMT shared task specifically aimed to address this by evaluating MT techniques that can effectively utilize multilingualism and monolingual data in the context of these diverse Indian languages. The paper tackles the problem of building a performant MT system for a wide array of Indic languages, aiming to improve translation quality despite resource limitations.

Methods

ANVITA-1.0 is built upon the Transformer architecture, a standard for modern NMT. The core innovation lies in its multilingual approach: two separate multilingual NMT models were trained, one for English-to-Indic directions and another for Indic-to-English directions. Both models share a common encoder-decoder structure, facilitating knowledge transfer across language pairs.

Key methodological contributions include:

  • Rigorous Data Filtering: Recognizing that Transformer models are sensitive to noisy data, the authors applied a comprehensive set of heuristics to filter the MultiIndicMT WAT 2021 corpora. This involved removing empty sentences, excessively long sentences, and pairs with skewed sentence length ratios.
  • Sub-word Tokenization: To handle the morphological complexity of Indian languages, SentencePiece was used for sub-word tokenization, creating a joint vocabulary for Indic languages and English.
  • Source Sentence Tagging: Special tokens were prepended to source sentences to explicitly indicate the language pair, aiding the model in multilingual settings.
  • Data Augmentation:
    • Related Language Transliteration: For the three lowest-resource languages (Oriya, Kannada, Gujarati), parallel data from related, higher-resource languages was transliterated and added.
  • Back Translation: Monolingual corpora were used to generate synthetic parallel data via back-translation, specifically for four low-resource languages (Oriya, Kannada, Punjabi, Gujarati).
  • Model Ensembling: The last five checkpoints of the trained models were ensembled to improve generalization.

Datasets

  • Training Data:
    • MultiIndicMT WAT 2021 Corpora: This was the primary source of parallel data, comprising approximately 10 million sentences across 10 Indian languages and English, covering 20 translation directions.
  • AI4Bharat Monolingual Corpora: Used for generating synthetic parallel data through back-translation.
  • Evaluation Data: The official test set provided by the WAT 2021 MultiIndicMT shared task organizers.

Key Insights and Ideas

  • Indic-to-English Outperforms English-to-Indic: The system consistently achieved better performance for Indic-to-English translation directions compared to English-to-Indic directions across all language pairs, as measured by BLEU and RIBES. This suggests that the "Many-to-One" mapping with a shared decoder might be more effective than "One-to-Many" for this multilingual setup.
  • Data Filtering is Crucial: The paper emphasizes that rigorous data filtering significantly improved translation performance and convergence rates, highlighting the sensitivity of Transformer models to noisy training data.
  • Data Augmentation Benefits Low-Resource Languages: Techniques like related language transliteration and back-translation were vital in boosting the performance for languages with limited parallel data.
  • Scoring Metric Matters: RIBES and AM-FM based scoring metrics placed ANVITA relatively better among participants compared to BLEU, suggesting these metrics might be more sensitive to the nuances of Indic language translation.

Key Results

ANVITA-1.0 achieved notable results on the WAT 2021 MultiIndicMT test set:

  • Highest AM-FM score for English→Bengali.
  • 2nd highest AM-FM score for English→Tamil.
  • 3rd highest AM-FM score for English→Hindi and Bengali→English.
  • Overall, performance for Indic-to-English directions was generally better than English-to-Indic directions, with BLEU scores ranging from 27.29 to 40.05 for Indic-to-English and 6.17 to 35.85 for English-to-Indic.

Limitations and Future Work

  • Limited Model Diversity for Ensembling: Due to time and resource constraints, the authors could not explore diverse models for ensembling, which could have further improved performance.
  • Potential for More Advanced Filtering: Despite extensive filtering, the training data was not entirely noise-free. Future work could explore more effective data filtering techniques.
  • Investigating Decoder Architectures: The authors suggest investigating whether separate decoders for each language pair, in conjunction with a shared encoder, could improve "One-to-Many" translation performance.
  • Handling Script Overlap: The paper notes that mapping Marathi characters to Unicode blocks to avoid script overlap with Hindi impacted results, suggesting this area needs further experimentation.
  • Need for More Diverse Parallel Corpora: The authors highlight the critical need for more diverse parallel corpora for Indian languages to advance Indic MT tasks and automated corpus compilation methods.

Summary updated

BibTeX
@inproceedings{vegi2021anvita,
  author = {Pavanpankaj Vegi and Sivabhavani J and Biswajit Paul and Chitra Viswanathan and Prasanna Kumar K R},
  title = {{ANVITA Machine Translation System for WAT 2021 MultiIndicMT Shared Task}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2021},
}
Figure