Exploiting Language Relatedness in Machine Translation Through Domain Adaptation Techniques

Amit Kumar, Rupjyoti Baruah, Ajay Pratap, Mayank Swarnkar, Anil Kumar Singh

TL;DR

Researchers have developed a novel data selection method called Scaled Similarity Score (SSS) and a transliteration trick to boost machine translation for low-resource languages like Nepali. By leveraging the linguistic closeness of Hindi and Nepali and filtering out-of-domain data more effectively, they achieved improvements of up to 3 BLEU points over standard techniques.

What Problem Does This Paper Address?

Machine Translation (MT) is a data-hungry field. While systems for English-German are world-class, "low-resource" languages (LRLs)—those without massive parallel corpora—struggle to reach usable quality.

A common strategy is Domain Adaptation (DA): taking a model trained on a general or "out-of-domain" dataset (like the Bible or technical manuals) and tweaking it for a specific "in-domain" task (like news or daily conversation). However, standard data selection methods often fail when data is extremely scarce. This paper tackles the specific challenge of translating between Hindi and Nepali, two closely related languages that share the Devanagari script, similar word orders, and many cognates, but lack the massive datasets required for traditional Neural Machine Translation (NMT).

Methods

The authors propose three key innovations to bridge the resource gap:

  1. WX-Transliteration for Language Modeling: Before training their filtering models, they convert the Devanagari script into WX-notation (a transliteration scheme for Indian languages). This reduces the "orthographic distance" between Hindi and Nepali, making it easier for a language model to recognize that the two languages are linguistically similar.
  2. Scaled Similarity Score (SSS): Traditional data selection often uses cross-entropy to find sentences in a large "out-of-domain" pool that look like "in-domain" data. The authors instead use SSS, which scales the language model scores between 0 and 1. This normalization helps the model favor sentences that are strictly closer to the target domain, filtering out noise more effectively.
  3. Integrated DA Techniques: They applied SSS across three common training workflows:
  4. Multi-Domain Approach (MDA): Training on a mix of domains.
  5. Fine-Tuning Approach (FTA): Training on out-of-domain data first, then specialized in-domain data.
  6. Iterative Back-Translation (SSSIBA): Using the model to translate monolingual text to create synthetic parallel data, but using SSS to filter only the best synthetic pairs.

Datasets

The study utilized several datasets across different domains:

  • In-domain: Hindi-Nepali parallel data from the WMT 2019 shared task (~65k sentences).
  • Out-of-domain: Agriculture (AGRI), Entertainment (ENT), and Bible (BIB) corpora sourced from TDIL and OPUS.
  • Monolingual: 90k sentences each for Hindi and Nepali to support back-translation.

Key Insights and Ideas

  • Transliteration is a "Cheat Code": By using WX-notation, the perplexity of their KenLM language model dropped drastically (e.g., from 477.3 down to 8.1 for Hindi). This suggests that for related languages, the script often masks how similar the underlying languages actually are.
  • Scaling Matters: Raw cross-entropy scores can have massive gaps between minimum and maximum values. Scaling these into the SSS (0 to 1 range) prevents outliers from dominating the selection process and ensures the model picks sentences that truly represent the target domain.
  • SMT Still Has Legs: Interestingly, the paper notes that Statistical Machine Translation (SMT) can sometimes outperform NMT on very limited datasets for these language pairs, likely because SMT handles orthographic similarity and word-for-word overlaps more robustly when data is too thin for deep learning.

Key Results

The proposed SSS method consistently outperformed the state-of-the-art data selection methods (like Moore-Lewis filtering):

  • Multi-domain: Gained ~2 BLEU points.
  • Fine-tuning: Gained ~3 BLEU points.
  • Iterative Back-Translation: Gained ~2 BLEU points.
  • Qualitative Similarity: The authors demonstrated that domains with higher "overlapping vocabulary" (like Entertainment) naturally performed better than those with low overlap (like the Bible), confirming that lexical similarity is a primary driver of MT quality in this setup.

Limitations and Future Work

The primary limitation is that this approach relies on language relatedness. It works beautifully for Hindi-Nepali because they share a script and origin; it might be less effective for unrelated language pairs (e.g., Hindi to English).

The authors suggest that future work should explore dynamic data selection—adjusting the filtering threshold during the training process—and expanding the technique to cross-lingual transfer learning involving multiple similar languages simultaneously (e.g., using Hindi and Marathi to improve Nepali translation).

Summary updated

Abstract

One of the significant challenges of Machine Translation (MT) is the scarcity of large amounts of data, mainly parallel sentence aligned corpora. If the evaluation is as rigorous as resource-rich languages, both Neural Machine Translation (NMT) and Statistical Machine Translation (SMT) can produce good results with such large amounts of data. However, it is challenging to improve the quality of MT output for low resource languages, especially in NMT and SMT. In order to tackle the challenges faced by MT, we present a novel approach of using a scaled similarity score of sentences, especially for related languages based on a 5-gram KenLM language model with Kneser-ney smoothing technique for filtering in-domain data from out-of-domain corpora that boost the translation quality of MT. Furthermore, we employ other domain adaptation techniques such as multi-domain, fine-tuning and iterative back-translation approach to compare our novel approach on the Hindi-Nepali language pair for NMT and SMT. Our approach succeeds in increasing ~2 BLEU point on multi-domain approach, ~3 BLEU point on fine-tuning for NMT and ~2 BLEU point on iterative back-translation approach.
BibTeX
@misc{kumar2023exploiting,
  author = {Amit Kumar and Rupjyoti Baruah and Ajay Pratap and Mayank Swarnkar and Anil Kumar Singh},
  title = {{Exploiting Language Relatedness in Machine Translation Through Domain Adaptation Techniques}},
  year = {2023},
  eprint = {2303.01793},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2303.01793},
}
Figure