Improving Low-Resource Machine Translation for Formosan Languages Using Bilingual Lexical Resources

Francis Zheng, Edison Marrese-Taylor, Yutaka MatsuoACL 2024 · OpenReview

Abstract

This paper investigates how machine translation for low-resource languages can be improved by incorporating information from bilingual lexicons during the training process. Focusing primarily on translation between Mandarin and 16 endangered Formosan languages, as well as Spanish-Nahuatl, we evaluate three integration strategies. Our results show that treating lexical data as additional parallel data, combined with pseudo-parallel sentence generation, yields the most significant gains. On average, this combined approach increased BLEU scores by 5.55 and chrF scores by 10.33, demonstrating that existing lexical resources are a powerful tool for bridging the digital divide in machine translation.

Introduction

Modern Neural Machine Translation (NMT) typically requires millions of parallel sentences to achieve high-quality results. This data-hungry nature leaves the majority of the world's 7,000 languages—including the indigenous Formosan languages of Taiwan—critically underserved. While these languages lack large-scale parallel corpora, many possess bilingual lexicons compiled by linguists for documentation. This research explores how to effectively leverage these dictionaries to "squeeze" the most translation utility out of overlooked lexical resources in extreme low-resource settings.

Methodology

The study evaluates three primary strategies for integrating bilingual lexicons into a standard Transformer-based NMT pipeline:

  1. Lexical Data as Parallel Data: Treating each word-translation pair in the lexicon as a short, one-word "sentence" and adding it directly to the training corpus.
  2. Pseudo-Parallel Data (PPD): Generating synthetic training data by replacing words in existing parallel sentences with their dictionary equivalents. A 20% replacement rate was found to be optimal.
  3. Combined Approach: Utilizing both raw lexical entries and augmented pseudo-parallel sentences simultaneously during training.

Datasets

The primary focus is on 16 Formosan languages, which are linguistically diverse and almost all moribund or critically endangered. The researchers compiled a dataset of parallel sentences and bilingual lexicons scraped and cleaned from the Indigenous Languages Research and Development Foundation in Taiwan.

Table 1: Statistics of parallel sentences and lexicon entries for 16 Formosan languages.

As shown in Table 1, the parallel sentence counts are notably low, with most languages having fewer than 10,000 sentences, highlighting the severity of the low-resource challenge compared to the relatively larger lexical resources available.

Key Results

The experiments demonstrate that incorporating lexical resources consistently improves translation quality across all tested languages and directions. The combined approach (Lexicon + PPD) emerged as the most effective strategy.

Table 2: Translation performance (BLEU and chrF) for baseline models versus lexicon-augmented models across all language pairs.

As detailed in Table 2, the augmented models significantly outperform the baselines. For instance, in the Amis-to-Mandarin task, the BLEU score jumped from 3.56 to 11.12, a gain of +7.56 points. On average, the techniques provided a 5.55 BLEU and 10.33 chrF point improvement. The methods also proved successful for the Spanish-Nahuatl pair, indicating generalizability across different language families.

Discussion and Insights

Several key findings emerged from the analysis:

  • Simplicity is Effective: Treating dictionary entries as parallel data is a surprisingly powerful way to boost performance, often outperforming more complex replacement-based techniques.
  • Vocabulary Expansion: Lexicons help the model learn rare terms that do not appear in limited parallel sentence data, effectively bridging the "rare word" gap.
  • Linear Scaling: Sensitivity studies revealed a strong positive correlation ($R^2$ often $> 0.9$) between the number of lexicon entries added and the resulting BLEU score, suggesting that even small additions to a dictionary yield measurable improvements.
  • Linguistic Diversity: Despite geographic proximity, Formosan languages are linguistically diverse. The authors found that multilingual pre-training provided minimal gains, suggesting that specialized individual models are currently more effective for these specific languages.

Conclusion

Bilingual lexicons are an underutilized but highly effective resource for improving NMT in low-resource scenarios. By integrating these dictionaries directly into the training process, substantial gains in translation quality can be achieved for endangered languages. Future work will explore incorporating contextual nuances to further refine these systems for sensitive cultural resources.

Summary updated

Abstract

This paper investigates how machine translation for low-resource languages can be improved by incorporating information from bilingual lexicons during the training process for mainly translation between Mandarin and Formosan languages, which are all moribund or critically endangered, and we also show that our techniques work for translation between Spanish and Nahuatl, a language pair consisting of languages from completely different language families. About 70% of the approximately 7,000 languages of the world have data in the form of lexicons, a valuable resource for improving low-resource language translation. We collect a dataset of parallel data and bilingual lexicons between Mandarin and 16 different Formosan languages and examine mainly three different approaches: (1) simply using lexical data as additional parallel data, (2) generating pseudo-parallel sentence data to use during training by replacing words in the original parallel sentence data using the lexicon, and (3) a combination of (1) and (2). All three approaches give us gains in both Bleu scores and chrF scores, and we found that (3) provided the most gains, followed by (1) and then (2), which we observed for both translation between Mandarin and the Formosan languages and Spanish-Nahuatl. With technique (3), we saw an average increase of 5.55 in Bleu scores and 10.33 in chrF scores.
BibTeX
@inproceedings{zheng2024improving,
  author = {Francis Zheng and Edison Marrese-Taylor and Yutaka Matsuo},
  title = {{Improving Low-Resource Machine Translation for Formosan Languages Using Bilingual Lexical Resources}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2024},
  url = {https://openreview.net/forum?id=K7J9eqmLL1},
}
Figure