Advancing NMT for Indigenous Languages: A Case Study on Yucatec Mayan and Chol

Julio Rangel, Norio KobayashiNAACL 2024

TL;DR

This paper tackles the challenge of building Neural Machine Translation (NMT) systems for low-resource Indigenous Mayan languages (Yucatec Mayan and Chol) by leveraging large language models (LLMs) pre-trained on Spanish. They demonstrate that multilingual models, particularly M2M100, show promising results, paving the way for improved accessibility and preservation of these languages.

What Problem Does This Paper Address?

Developing effective Natural Language Processing (NLP) tools for Indigenous languages of the Americas is a critical but often overlooked area. Many of these languages, despite having deep linguistic roots and significant speaker populations, lack the extensive parallel corpora needed to train robust machine translation systems. This scarcity hinders their accessibility to essential services like healthcare and legal information, and impedes their digital preservation. This paper aims to bridge this gap by exploring how pre-trained Spanish LLMs can be adapted for NMT tasks involving Yucatec Mayan and Chol, two Mayan languages.

Methods

The researchers employed a prompt-based approach to train both one-to-many and many-to-many NMT models. They experimented with two powerful LLMs: T5S (a Spanish-centric T5 variant) and M2M100 (a multilingual translation model). The core idea was to adapt these models, pre-trained on vast amounts of Spanish data, to translate to and from Yucatec Mayan and Chol. They explored different training configurations, including training on individual languages, combined Mayan and Chol datasets, and augmenting with additional data from the AmericasNLP 2023 Shared Task.

Datasets

The primary dataset used in this study comprised 28,135 translation pairs extracted from the CPLM (Parallel Corpus for Mexican Languages) web tools, specifically for Spanish-to-Chol and Spanish-to-Yucatec Mayan. This dataset was compiled and processed, with pairs filtered based on character length ratios to ensure quality. Additionally, they incorporated data from the AmericasNLP 2024 Shared Task, which provided parallel data for 11 other Indigenous languages.

Key Insights and Ideas

  • Leveraging Spanish LLMs is Viable: The study confirms that large language models pre-trained on Spanish can be effectively adapted for low-resource Indigenous languages, overcoming the data scarcity challenge.
  • Multilingual Models Shine: The M2M100 model, designed specifically for translation, generally outperformed the more general-purpose T5S model, highlighting the benefits of specialized architectures for this task.
  • Combined Datasets Offer Advantages: Training models on combined Yucatec Mayan and Chol datasets (one-to-many) yielded better average performance than training on individual languages, suggesting a synergistic effect.
  • Data Augmentation Strategies Matter: While augmenting with additional data from the AmericasNLP task initially decreased performance, implementing a uniform sampling strategy mitigated this degradation, especially when training across multiple datasets.
  • Bidirectional Training Potential: For T5S, bidirectional training (translating in both directions) showed promise, potentially due to its flexible design that doesn't strictly require explicit language tags.

Key Results

The best-performing configurations achieved ChrF++ scores around 50 and BLEU scores in the high 20s for M2M100 on the test sets. For instance, the "Mayan and Chol" configuration for M2M100 achieved a ChrF++ of 48.88 and a BLEU of 25.11 on the test set. T5S, while generally lower, still showed competitive results, with the "Mayan and Chol" configuration reaching a ChrF++ of 35.1 and a BLEU of 11.42 on the test set. The "Zero shot" configurations for both models performed significantly lower, underscoring the importance of fine-tuning.

Limitations and Future Work

The authors acknowledge that the current dataset, while a good starting point, is still relatively small. They also note that the "biAmeNLP + uniform" strategy did not yield optimal results for M2M100, possibly due to the need for target language tags, which are not always explicitly available for Indigenous languages. Future work will focus on expanding datasets, exploring active learning and few-shot learning techniques, customizing M2M100 for other Indigenous languages, and investigating tag selection for improved translation accuracy. They also plan to apply these techniques to other low-resource scenarios, such as natural language to SPARQL translation.

Summary updated

BibTeX
@inproceedings{rangel2024advancing,
  author = {Julio Rangel and Norio Kobayashi},
  title = {{Advancing NMT for Indigenous Languages: A Case Study on Yucatec Mayan and Chol}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2024},
}
Figure