Unsupervised Morphological Segmentation for Low-Resource Polysynthetic Languages

Ramy Eskander, Judith L. Klavans, Smaranda Muresan2019 · DOI

Abstract

Polysynthetic languages pose a challenge for morphological analysis due to the root-morpheme complexity and to the word class "squish". In addition, many of these polysynthetic languages are low-resource. We propose unsupervised approaches for morphological segmentation of low-resource polysynthetic languages based on Adaptor Grammars (AG). We experiment with four languages from the Uto-Aztecan family. Our AG-based approaches outperform other unsupervised approaches and show promise when compared to supervised methods, outperforming them on two of the four languages.

Introduction

Morphological segmentation—the process of decomposing words into their smallest meaningful units, or morphemes—is a fundamental task in natural language processing. However, this task is exceptionally difficult for polysynthetic languages, such as those in the Uto-Aztecan family. These languages are characterized by "holophrasis," where a single complex word can convey the meaning of an entire sentence in languages like English. Furthermore, they exhibit "word class squish," where boundaries between nouns, verbs, and adjectives are often blurred.

Compounding these linguistic challenges is the fact that most polysynthetic languages are low-resource. There is often insufficient human-labeled data to train the large-scale supervised neural models that currently dominate the field. This research explores the efficacy of unsupervised Adaptor Grammars (AGs) to bridge this gap, learning language structure directly from raw text.

Methodology

The study utilizes Adaptor Grammars, which are nonparametric Bayesian models that extend Probabilistic Context-Free Grammars (PCFGs) with a "memory" mechanism. While a standard PCFG learns the probabilities of individual rules, an AG can learn and adapt to specific, frequently occurring subtrees, effectively memorizing entire morpheme sequences.

The researchers evaluated five distinct AG configurations:

  1. Language-Independent (LIMS): A "best-on-average" grammar configuration derived from previous research on unrelated languages (e.g., Turkish, Finnish).
  2. Best Configuration per Language (BestL): A grammar structure specifically tuned for each target language.
  3. Seeded Knowledge (Scholar): A semi-supervised approach providing the model with the most frequent prefixes and suffixes from the training set.
  4. Multilingual Training: A joint model trained on all four related languages to exploit shared structural patterns.
  5. Data Augmentation (Aug): Incorporating unsegmented text from Bible translations to increase the available raw data.

Datasets

The experiments focused on four Uto-Aztecan languages: Mexicanero (MX), Nahuatl (NH), Wixarika (WX), and Yorem Nokki (YN). The primary datasets consisted of small vocabularies ranging from approximately 400 to 700 words for training. For Nahuatl and Wixarika, larger unsegmented datasets from New Testament translations were also utilized for the data augmentation experiments.

Results and Discussion

The AG-based approaches demonstrated significant improvements over standard unsupervised baselines, such as Morfessor and MorphoChain, across all four languages.

Comparison of Adaptor Grammar (AG) variants against unsupervised baselines Morfessor and Morphochain.

One of the most notable findings was the competitive performance of AGs against supervised models. In low-resource settings, these "smart" unsupervised models can actually surpass supervised alternatives. For Nahuatl and Yorem Nokki, the best AG configurations achieved higher F1-scores than supervised Sequence-to-Sequence (S2S) and Conditional Random Field (CRF) models.

Performance comparison between the best Adaptor Grammar (BestAG) and supervised models (S2S, CRF, BestMTT, BestDA).

Key Insights

  • Structural Universals: The success of the "LIMS" setup suggests that polysynthetic languages share underlying structural "skeletons" with other morphologically rich languages that AGs can effectively exploit.
  • Ambiguity Challenges: Error analysis indicated that morpheme ambiguity—where a character sequence may or may not function as a morpheme depending on context—is a primary source of error. This is particularly evident in Wixarika, which features many one-letter morphemes.
  • Oversegmentation: The models occasionally exhibit a tendency to break single morphemes into smaller, meaningless units, particularly when those units resemble common affixes.

Conclusion

This work demonstrates that unsupervised Adaptor Grammars are a powerful tool for the morphological analysis of low-resource polysynthetic languages. By outperforming supervised models in certain contexts, AGs offer a viable path forward for language preservation and NLP development without the need for extensive, expensive human annotation. Future research will focus on improving the handling of short morphemes and exploring more advanced cross-lingual transfer techniques.

Summary updated

Abstract

Polysynthetic languages pose a challenge for morphological analysis due to the rootmorpheme complexity and to the word class "squish". In addition, many of these polysynthetic languages are low-resource. We propose unsupervised approaches for morphological segmentation of low-resource polysynthetic languages based on Adaptor Grammars (AG) We experiment with four languages from the Uto-Aztecan family. Our AG-based approaches outperform other unsupervised approaches and show promise when compared to supervised methods, outperforming them on two of the four languages.
BibTeX
@misc{eskander2019unsupervised,
  author = {Ramy Eskander and Judith L. Klavans and Smaranda Muresan},
  title = {{Unsupervised Morphological Segmentation for Low-Resource Polysynthetic Languages}},
  year = {2019},
  doi = {10.18653/v1/w19-4222},
}
Figure