TL;DR
This paper presents a new, open-source finite-state morphological analyzer for Highland Puebla Nahuatl, an endangered Uto-Aztecan language. The tool achieves high coverage and precision, making it a valuable resource for linguistic research, language learning, and the preservation of this language.
What Problem Does This Paper Address?
Highland Puebla Nahuatl, like many polysynthetic and agglutinating languages, has a complex morphology with heavy prefixing and suffixing. This complexity makes it challenging to determine the base stem of words, a crucial step for tasks like corpus annotation, spell-checking, and electronic dictionary lookup. Existing computational tools for Nahuatl are often not freely available or easily accessible, creating a gap in resources for researchers and language advocates. This paper aims to fill that gap by developing a robust, open-source morphological analyzer for this specific Nahuatl variant.
Methods
The authors leverage the Helsinki Finite-State Toolkit (HFST) to build their morphological analyzer. They employ the lexc formalism for modeling the language's morphotactics (how morphemes are ordered) and the twol formalism for handling morphophonological alternations (sound changes that occur at morpheme boundaries). The process involves creating a lexicon of approximately 5,000 lexemes, categorized by part of speech and added in frequency order. They also developed a tagset based on the Apertium project's conventions, with a conversion to Universal Dependencies for broader compatibility. A separate finite-state transducer is used to manage orthographic and spelling variations, which are common in Nahuatl.
Datasets
The data used to develop the analyzer comes from three sources:
- A dataset of transcribed recordings of interviews and conversations, primarily about plants.
- A subset of texts from the
azzvariant of the multi-variant parallel corpus Axolotl. - Technical publications from the Sociedad Mexicana de Física, which include translations of scientific texts.
The paper does not introduce a new benchmark dataset but rather uses existing corpora for development and evaluation.
Key Insights and Ideas
- The Power of Finite-State Transducers for Complex Morphology: The paper demonstrates that finite-state technology, despite its "classical" perception, remains highly effective for modeling the intricate morphology of polysynthetic languages like Highland Puebla Nahuatl.
- Addressing Orthographic Variation is Crucial: Nahuatl exhibits significant orthographic variation, even within the work of a single author. The authors highlight the necessity of a dedicated system to handle these variations to achieve good coverage.
- Open-Source is Key for Endangered Languages: By releasing their analyzer as open-source, the authors contribute to a growing movement to provide accessible computational tools for endangered languages, fostering further research and development.
- Relational Nouns Present Unique Challenges: The paper details the specific challenges posed by relational nouns in Nahuatl, which have obligatory possession and can be compounded or used with free-standing complements, requiring careful modeling.
Key Results
The developed finite-state morphological transducer demonstrates:
- Reasonable Coverage: Approximately 90% coverage on freely available corpora of the language.
- High Precision: Over 95% precision on a manually verified test set.
- Lexicon Size: The stem lexicon contains around 5,043 entries, covering verbs, nouns, adverbs, adjectives, proper nouns, and other categories.
- Morphophonological Rules: A limited set of
twolrules are used, primarily for degemination and nasal assimilation.
Limitations and Future Work
The authors acknowledge several limitations and outline future directions:
- Lexicon Expansion: The lexicon could be expanded to include more stems, particularly for verb forms, as over half of unknown words were verbs.
- Increased Corpus Coverage: The goal is to increase coverage across all corpora and to obtain new corpora for testing.
- Compounding and Incorporation: Future work will focus on including support for compounding and incorporation, which are significant features of Nahuatl morphology.
- Weighting the Transducer: The authors have manually disambiguated 10,000 tokens and plan to use this data to weight the transducer, prioritizing more probable analyses.
- Annotated Data for Evaluation: The current evaluation relies on manual inspection due to the lack of a large, annotated dataset. Developing such a dataset would allow for more rigorous evaluation of precision and recall.
Summary updated