TL;DR
This paper introduces the first syntactically annotated corpus for K'iche', a Mayan language, using Universal Dependencies. This resource enables the development of NLP tools for K'iche' and other Mayan languages, addressing a significant gap in computational linguistics for under-resourced languages.
What Problem Does This Paper Address?
Developing natural language processing (NLP) tools for any language, especially those with complex morphology and syntax, requires large, annotated datasets. For many indigenous languages, such as K'iche', these resources are scarce or non-existent, hindering research and technological development. While some K'iche' data exists, it's often not in a format suitable for modern NLP pipelines, which rely on standardized treebanks. This paper aims to bridge this gap by creating a K'iche' corpus annotated according to the Universal Dependencies (UD) framework, making it compatible with a wide range of existing NLP tools and facilitating cross-linguistic research.
Methods
The authors built a corpus of K'iche' sentences, totaling 1,433 sentences and approximately 10,000 tokens. The annotation process followed the Universal Dependencies guidelines, focusing on morphosyntactic structure. Key steps included:
- Data Collection: Sentences were gathered from various sources, including dictionaries, language learning materials, religious texts, legal texts, and folk tales, with translations provided for most.
- Preprocessing: A freely available morphological analyzer was used to generate potential analyses for each token. These were then disambiguated manually, and missing analyses were added.
- Annotation: The data was converted to the CoNLL-U format. Morphological tags were mapped to Feature=Value pairs using a deterministic algorithm. The annotation itself was performed by the first author, with difficult cases discussed with the second author.
- Linguistic Analysis: The paper details the annotation of specific K'iche' constructions, such as relational nouns, nominal possession, relative clauses, non-verbal predicates, complement clauses, and valency-changing operations (passive and antipassive). These sections highlight how K'iche' phenomena were mapped onto the UD framework, sometimes requiring adaptations or extensions.
Datasets
The primary dataset introduced is a K'iche' corpus consisting of 1,433 sentences and approximately 10,000 tokens. This corpus is derived from:
- Published dictionaries (Medrano Rojas, 2004)
- Linguistic research (Can Pixabaj, 2015; Henderson, 2012)
- Language learning materials (Romero et al., 2018)
- Religious, medical, and legal texts (Wycliffe Bible Translators, 2011; Wikimedia Incubator, 2017; Méndez López, 2020; Gobierno de Guatemala, 2009)
- Folk tales (Ministerio de Educación, 2016a,b)
This corpus is made available under a free/open-source license. The paper also mentions using the Crúbadán corpus of K'iche' (Scannell, 2007) for a larger-scale linguistic example mining experiment.
Key Insights and Ideas
- UD as a Unifying Framework: The paper demonstrates the applicability of the Universal Dependencies framework to a Mayan language, showcasing its flexibility in capturing diverse morphosyntactic structures.
- K'iche' Morphosyntax: The work provides detailed insights into specific K'iche' grammatical features, such as ergative-absolutive alignment, verb-initial order, agreement markers, relational nouns, and status suffixes, and how these are represented in the annotation.
- Utility for NLP: The creation of this treebank is crucial for developing NLP tools for K'iche', enabling tasks like machine translation, relation extraction, and parsing.
- Linguistic Example Mining: The corpus is shown to be valuable for testing linguistic hypotheses and discovering counterexamples to existing generalizations, as demonstrated by the analysis of status suffixes.
- First K'iche' Treebank: This represents the first syntactically annotated treebank for K'iche', a significant contribution to the field of indigenous language NLP.
Key Results
- Corpus Creation: A corpus of 1,433 K'iche' sentences, annotated for morphosyntax according to Universal Dependencies, was successfully created and released.
- Parsing Evaluation: Experiments with three NLP pipelines (UDPipe 1.2, UDPipe 2.0, and UDify) showed that UDPipe 2.0 performed best, achieving F1 scores of 76.7% for LAS (Labeled Attachment Score) on the K'iche' corpus.
- Linguistic Discovery: The corpus facilitated the discovery of potential counterexamples to existing linguistic generalizations regarding phrase-final status suffixes, highlighting the value of annotated data for linguistic research.
Limitations and Future Work
- Limitations:
- The authors acknowledge that the corpus size, while substantial for an under-resourced language, might not be sufficient for training highly complex models without careful cross-validation.
- The annotation of some constructions, particularly those with complex agreement or derivational morphology, required careful consideration and potential adaptations of UD guidelines.
- Future Work:
- Investigate the use of enhanced dependencies to provide a more semantics-oriented encoding of relational nouns.
- Expand the treebank and apply the learned annotation solutions to other Mayan languages, such as Uspanteko and Kaqchikel.
- Further explore the use of the corpus for linguistic example mining and hypothesis testing.
Summary updated