Proceedings of the First Workshop on Natural Language Processing for Indigenous Languages of the Americas

Manuel Mager, Arturo Oncevay, Annette Rios, Ivan Vladimir Meza Ruiz, Alexis Palmer, Graham Neubig, Katharina KannNAACL 2021

TL;DR

This document presents the proceedings of the first AmericasNLP workshop, highlighting the critical need and growing efforts to develop NLP tools and resources for Indigenous languages of the Americas. It showcases diverse research tackling challenges like data scarcity, morphological complexity, and the creation of new datasets and models, underscoring the importance of linguistic diversity in NLP.

What Problem Does This Paper Address?

The field of Natural Language Processing (NLP) has seen tremendous advancements, largely driven by the availability of vast amounts of digital text and speech data for widely spoken languages. However, this progress has not extended equally to the thousands of Indigenous languages of the Americas, which are often severely under-resourced. These languages face unique challenges, including:

  • Data Scarcity: Limited digital text and speech data for training NLP models.
  • Linguistic Diversity: A wide range of grammatical structures, including highly polysynthetic languages with complex morphology, which differ significantly from the analytical structures common in Indo-European languages.
  • Endangerment: Many Indigenous languages are endangered, making the preservation and revitalization of these languages through technology an urgent concern.

This workshop aims to bring together researchers working on these challenges, fostering collaboration and advancing the state-of-the-art in NLP for Indigenous languages of the Americas.

Methods

The workshop proceedings showcase a variety of NLP methods applied to Indigenous languages, including:

  • Corpus Creation and Annotation: Development of new annotated corpora for tasks like coreference resolution, morphosyntactic analysis, and part-of-speech tagging. Examples include corpora for Conchucos Quechua, K'iche', and Yoloxóchitl Mixtec.
  • Machine Translation (MT): Building MT systems, often leveraging cross-lingual transfer learning and pre-trained models, for language pairs such as Ayuuk-Spanish, Spanish-Quechua, and for low-resource South American languages.
  • Speech Recognition (ASR): Developing ASR systems, with a focus on improving performance in low-resource settings, including investigations into explicit tone transcription for Bribri.
  • Morphological Analysis and Segmentation: Creating finite-state transducers and neural models for morphological analysis and segmentation, particularly for polysynthetic languages like Seneca and Gitksan.
  • Lexical Resource Development: Building dictionaries and lexical resources, sometimes using techniques like neural machine translation to reconstruct lexicons from related languages.
  • Universal Dependencies (UD) Treebanks: Extending the UD framework to polysynthetic languages, exemplified by work on St. Lawrence Island Yupik.
  • Language Modeling: Utilizing character-based language models for tasks like investigating variation in written forms of Nahuatl.
  • Embedding Techniques: Leveraging English word embeddings for semi-automatic semantic classification in Nêhiyawêwin (Plains Cree).
  • OCR for Lexical Resources: Using OCR to create lexical resources from historical documents, as demonstrated for Ancash.

Datasets

The workshop features research utilizing and introducing several datasets:

  • Newly Created/Annotated Corpora:
    • qxoRef 1.0: A coreference corpus for Conchucos Quechua.
  • K'iche' Corpus: Annotated for morphosyntactic structure.
  • Yoloxóchitl Mixtec Corpus: Used for end-to-end ASR research.
  • Highland Puebla Nahuatl Speech Translation Corpus: For endangered language documentation.
  • Ancash Lexical Resources: Derived from OCR of historical documents.
  • Apurinã Universal Dependencies Treebank: A new UD treebank.
  • Western Tlacolula Valley Zapotec Data: For morphological transducer development.
  • Plains Cree / nêhiyawêwin Dictionary: Used for semantic classification.
  • Existing Datasets (adapted/used):
    • Various corpora for Guarani, Inuinnaqtun, and other Indigenous languages are used for specific tasks.
  • Shared Task datasets for Open Machine Translation and other NLP tasks are central to many contributions.

Key Insights and Ideas

  • The Urgency of Linguistic Preservation: The workshop strongly emphasizes that NLP is a crucial tool for the documentation, revitalization, and preservation of endangered Indigenous languages.
  • Morphological Complexity is a Major Hurdle: Many Indigenous languages exhibit rich and complex morphology, posing significant challenges for standard NLP techniques that often assume more analytical structures. This necessitates specialized approaches for morphological analysis and segmentation.
  • Cross-Lingual Transfer is Key: Given data scarcity, techniques like cross-lingual transfer learning, using pre-trained models from high-resource languages, and leveraging related languages are vital for building functional NLP systems.
  • Community Involvement is Essential: While not explicitly detailed in every paper, the underlying motivation for much of this work is the need to empower Indigenous communities with tools to manage and utilize their own languages.
  • The AmericasNLP Shared Task as a Catalyst: The shared task provided a crucial benchmark and a focal point for research, driving progress and enabling comparative analysis of different approaches for a range of Indigenous languages.

Key Results

The proceedings showcase a range of results across various tasks:

  • Machine Translation: Several teams presented systems for the AmericasNLP 2021 Shared Task, demonstrating varying degrees of success in translating between Indigenous languages and English/Spanish. Specific performance metrics are detailed within individual papers.
  • Speech Recognition: Research on Bribri showed that explicit tone transcription can significantly improve ASR performance in extremely low-resource settings.
  • Morphological Analysis: New morphological analyzers and segmenters were developed for languages like Seneca, Gitksan, and Paraguayan Guaraní, demonstrating the feasibility of applying advanced techniques to complex morphology.
  • Dataset Creation: The workshop saw the introduction of several new datasets, including UD treebanks and annotated corpora, which are critical for future research.

Limitations and Future Work

  • Data Scarcity Remains a Primary Challenge: Despite efforts, the lack of large-scale, high-quality datasets for most Indigenous languages continues to be a significant limitation.
  • Generalizability of Models: Models trained on one Indigenous language may not generalize well to others due to the vast linguistic diversity.
  • Evaluation Metrics: Developing appropriate evaluation metrics that accurately reflect the linguistic nuances and communicative goals for these languages is an ongoing area of work.
  • Future Work:
    • Continued development of new datasets and annotation tools.
  • Exploration of more advanced cross-lingual and few-shot learning techniques.
  • Integration of linguistic expertise and community input into NLP model development.
  • Building NLP tools that support language revitalization and educational efforts.
  • Expanding research to cover a wider range of Indigenous languages and NLP tasks.

Summary updated

BibTeX
@inproceedings{mager2021proceedings,
  author = {Manuel Mager and Arturo Oncevay and Annette Rios and Ivan Vladimir Meza Ruiz and Alexis Palmer and Graham Neubig and Katharina Kann},
  title = {{Proceedings of the First Workshop on Natural Language Processing for Indigenous Languages of the Americas}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2021},
}
Figure