TL;DR
This paper introduces the first parallel corpora for machine translation between Spanish and two indigenous Mexican languages: Mazatec and Mixtec. They demonstrate that fine-tuning pre-trained multilingual models, particularly M2M100-48, yields the best results, highlighting the importance of targeted approaches for low-resource languages.
What Problem Does This Paper Address?
Natural Language Processing (NLP) has seen tremendous growth, but this progress is heavily skewed towards high-resource languages. Machine Translation (MT), a key NLP application, excels for language pairs with abundant data (e.g., English-Spanish). However, for low-resource languages, especially indigenous ones, the lack of parallel datasets severely hinders MT performance. Mexico, a highly multilingual country, has many indigenous languages facing endangerment, and a lack of digital resources exacerbates this issue, potentially leading to cultural marginalization. This paper addresses the critical need for MT resources for indigenous languages by creating and evaluating parallel corpora for Mazatec and Mixtec.
Methods
The authors employed three main approaches to evaluate their newly created parallel corpus:
- Transformer: A standard Transformer architecture was used as a baseline, representing a common state-of-the-art approach for MT.
- Transfer Learning: This involved using pre-trained models (specifically, English-Spanish models from Hugging Face) and fine-tuning them on the low-resource language pairs. The idea is to leverage knowledge learned from a high-resource language to bootstrap performance on a low-resource one.
- Fine-tuning Pre-trained Multilingual Models: This was the core of their experimental setup. They utilized two prominent multilingual models: M2M100-48 and mBART50. These models are pre-trained on vast amounts of text across many languages and are then adapted to the specific Mazatec-Spanish and Mixtec-Spanish translation tasks.
Datasets
The primary contribution is the creation of new parallel datasets for:
- Mazatec-Spanish: Consisting of 9,799 sentences, sourced from religious texts, constitutions, and textbooks.
- Mixtec-Spanish: Consisting of 13,235 sentences, also sourced from religious texts, constitutions, and textbooks.
These datasets were pre-processed by removing numbers and special characters before being split into training, development, and test sets (70:10:20 ratio). The paper also mentions using English-Spanish models from the OPUS dataset for transfer learning experiments.
Key Insights and Ideas
- The critical role of parallel corpora: The study strongly emphasizes that the availability of parallel data is paramount for effective MT, especially for low-resource languages.
- Indigenous languages as target languages perform better: The results consistently showed better performance when translating into Mazatec or Mixtec compared to translating from them into Spanish. This suggests that Spanish-to-indigenous language translation is a less complex task for current models.
- Fine-tuning multilingual models is key: Pre-trained multilingual models, when fine-tuned, significantly outperform standard Transformer models and even transfer learning approaches for these low-resource languages.
- Model size matters in transfer learning: For transfer learning, a larger model configuration (transformer-big) yielded better results than a smaller one (transformer-base), indicating that model capacity plays a role.
- Domain-specific and cultural nuances are challenging: All tested approaches struggled with translating domain-specific and culturally specific terms, highlighting a persistent challenge in MT for indigenous languages.
Key Results
The paper reports BLEU scores for various configurations:
- Fine-tuning M2M100-48: Achieved the highest scores, with:
- Mazatec-Spanish: 12.09
- Spanish-Mazatec: 22.25
- Mixtec-Spanish: 16.75
- Spanish-Mixtec: 22.15
- Transformer baseline: Generally showed sub-optimal results, with scores significantly lower than fine-tuning approaches.
- Transfer learning: Performed better than the Transformer baseline but was outperformed by fine-tuning.
Limitations and Future Work
- Dataset Size: While the created datasets are significant for these languages, they are still considered small compared to high-resource language datasets, which impacts Transformer performance.
- Domain Specificity: The models struggled with domain-specific and culturally nuanced vocabulary, indicating a need for more specialized data or techniques.
- Future Work: The authors plan to explore advanced techniques like zero-shot and few-shot learning. These methods are promising for leveraging sparse data and pre-existing knowledge to further improve MT performance for underrepresented linguistic communities. They also aim to develop more robust translation systems.
Summary updated