TL;DR
This paper introduces the first Universal Dependencies (UD) treebank for St. Lawrence Island Yupik, a polysynthetic language. By annotating at the morpheme level, they demonstrate that this approach is crucial for accurately capturing the complex syntactic relationships in such languages, outperforming word-level annotation and paving the way for better NLP tools for under-represented languages.
What Problem Does This Paper Address?
The Universal Dependencies (UD) project aims to create a consistent, cross-lingual framework for annotating syntactic dependencies in natural language. However, the UD framework, largely designed for analytic languages, struggles with highly polysynthetic languages like St. Lawrence Island Yupik. These languages form words by combining many morphemes, leading to extremely long words that can encapsulate entire sentences. Traditional word-level annotation in UD often results in "degenerate trees" for these languages, failing to capture the rich syntactic information embedded within individual morphemes. This under-representation of polysynthetic languages in UD limits the development of NLP tools for these linguistic communities.
Methods
The core innovation of this paper is the development of a morpheme-level annotation scheme for St. Lawrence Island Yupik, in addition to the standard word-level annotation. They utilized an existing Yupik finite-state morphological analyzer to segment words into morphemes. Crucially, they treated each morpheme as an independent syntactic token, including derivational and inflectional morphemes. This required creating new dependency relations (e.g., dep:aux, dep:mark, dep:cop) and sub-relations (e.g., obl:mod, nmod:arg) to capture nuances not addressed by the standard UD guidelines. They also developed a script to convert their morpheme-level annotations to word-level annotations to maintain compatibility with existing UD tools.
Datasets
The primary dataset used is a corpus of exercise sentences from Jacobson's (2001) "A Practical Grammar of the St. Lawrence Island / Siberian Yupik Eskimo Language." This corpus was manually annotated at both the word and morpheme levels. For evaluation, they also used a separate test corpus extracted from Nagai (2001), which represents an elder Yupik speaker's speech and includes code-switching with English, serving as an out-of-domain test set.
Key Insights and Ideas
- Morpheme-level annotation is essential for polysynthetic languages: The paper strongly argues that treating morphemes as the basic unit of syntactic analysis is not just beneficial but necessary for accurately representing the syntax of languages like Yupik.
- Word-level annotation leads to information loss: Degenerate trees and failure to capture crucial syntactic relations are direct consequences of adhering strictly to word-level annotation for polysynthetic languages.
- UD guidelines need adaptation for polysynthesis: While UD provides a general framework, specific adaptations, including new dependency relations and treating morphemes as tokens, are required to accommodate the unique structures of polysynthetic languages.
- Morphological analyzers are key enablers: The availability of robust morphological analyzers is a critical prerequisite for developing morpheme-level UD treebanks for these languages.
Key Results
- Improved Parsing Accuracy: Morpheme-level parsing, when provided with correct morpheme segmentation (gold segmentation), significantly outperformed word-level parsing, achieving higher Unlabelled Attachment Score (UAS) and Labelled Attachment Score (LAS). For example, UAS reached 91.82% with morpheme-level gold segmentation compared to 88.86% for word-level.
- Impact of Segmentation Errors: Automatic morpheme segmentation introduced errors that severely hampered morpheme-level parsing performance when not using gold segmentation. This highlights the importance of accurate morphological analysis.
- Qualitative Analysis: The morpheme-level annotation successfully captured complex syntactic relations, such as the subject of embedded verbs and the role of derivational suffixes in forming verbal stems, which were lost in word-level annotations.
Limitations and Future Work
The authors acknowledge that their morpheme-level annotation scheme requires more linguistic resources, particularly a morphological analyzer. They also note that automatic morpheme segmentation can introduce errors that negatively impact parsing performance. Future work could explore treating only some derivational morphemes as syntactic tokens, rather than all morphemes, to potentially balance linguistic accuracy with annotation effort. The paper also suggests that the UD project should formally acknowledge the need to treat at least some derivational morphemes as syntactic tokens to improve its universality.
Summary updated