Unsupervised morphological segmentation in a language with reduplication

Simon Todd, Annie Huang, Jeremy Needle, Jennifer Hay, Jeanette KingNAACL 2022

TL;DR

This paper tackles the challenge of unsupervised morphological segmentation for languages with reduplication, a common but often overlooked linguistic process. By extending the Morfessor model with abstract reduplication templates, the authors significantly improve its ability to identify morpheme boundaries in Māori, demonstrating a promising approach for handling this typologically frequent phenomenon.

What Problem Does This Paper Address?

Unsupervised morphological segmentation aims to break down words into their smallest meaningful units (morphemes) without relying on pre-defined rules or dictionaries. This is valuable because it mimics how humans learn language and allows for broad applicability across languages. However, existing unsupervised models, like Morfessor, often make assumptions tailored to high-resource languages, potentially overlooking typologically common processes like reduplication. Reduplication, where a part of a word is repeated (e.g., "walk-walk" for emphasis), is a widespread phenomenon but computationally challenging to model. Standard Morfessor struggles with reduplication because it treats all morphemes as simple concatenations, failing to recognize the systematic repetition and its specific structural constraints. This leads to poor segmentation of reduplicated words and can even hinder the identification of other morphemes within them.

Methods

The core contribution is the extension of the Morfessor baseline model to explicitly handle reduplication. The authors introduce the concept of abstract reduplication templates. These templates are not tied to specific phonological forms but rather capture the structural patterns of reduplication (e.g., copying the whole base, copying the first syllable).

Key methodological points include:

  • RED Morpheme: A special, phonologically empty morpheme, RED, is added to Morfessor's lexicon. When reduplication is identified, the reduplicated part is replaced by RED, allowing the model to pool statistical support across all reduplication instances.
  • Manually-Defined Templates: The authors define three mutually-exclusive templates for Māori reduplication, based on linguistic knowledge of its structure (e.g., full reduplication, left-copying, right-copying). These templates are "loosely specified" to allow for arbitrary copying in other languages.
  • Integration into Training (RS): The recursive splitting (RS) algorithm used for training Morfessor is extended. When RS considers splitting a word, it now also evaluates an analysis involving reduplication (replacing a potential reduplicant with RED). This analysis is chosen if it leads to a lower cost.
  • Integration into Testing (Viterbi): For words not seen during training, the Viterbi algorithm is used. This is extended by adding a "vertical" dimension to consider partial analyses matching reduplication templates alongside the standard "horizontal" segmentation.
  • Base Independence Constraint: For Māori, an assumption is made that the base of reduplication must be morphologically simplex. This prevents further segmentation within an identified base.

Datasets

  • Training Data: A set of 19,595 word types from the Te Aka Māori Dictionary (Moorfield, 2011), including headwords and their inflections. Proper nouns were removed.
  • Test Data: A set of 1,889 word types categorized by Keegan (1996) as clear instances of reduplication. This dataset was curated to focus on reduplication with morphologically simplex bases, aiming to isolate the model's performance on this specific phenomenon. 83.5% of these test items were present in the Te Aka Dictionary.

Key Insights and Ideas

  • Reduplication is a "Free Lunch" for Segmentation: Explicitly modeling reduplication significantly boosts performance, not just for reduplicated words but also for general segmentation. This is attributed to the increased statistical support for the RED morpheme, which helps disambiguate other morphological structures.
  • Templates are Crucial, but Interactions Matter: While adding more templates generally improves performance, their interactions can lead to "coercion," where one template dominates and misclassifies items that should be handled by another. This highlights the need for careful template design.
  • Full Reduplication is Salient but Can Be Problematic: The "full reduplication" template (copying the entire base) is highly effective and often correctly identified. However, it can also "coerce" other types of reduplication (like left-2 and right) into its pattern, leading to errors.
  • Training Data Matters for Viterbi: Coercion to the full-reduplication template is more prevalent when test items are not seen during training and rely on the Viterbi algorithm. This suggests that a comprehensive training set, including bases of reduplication, is crucial for robust performance.
  • Linguistic Knowledge Enhances Unsupervised Models: This work strongly supports the idea that incorporating expert linguistic knowledge (like reduplication templates) into unsupervised models can lead to substantial improvements, bridging the gap between purely data-driven and rule-based approaches.

Key Results

  • The extended model (with all reduplication templates) achieved significantly higher performance than the original Morfessor (no templates).
  • Accuracy: Original: 0.23, Extended: 0.83
    • Recall: Original: 0.34, Extended: 0.98
    • Precision (Prec.0): Original: 0.28, Extended: 0.91
    • Precision (Prec.1): Original: 0.59, Extended: 0.91
  • The extended model drastically reduced the number of words left completely unsegmented (from 30.2% to 0.7%).
  • For words with simplex bases, the extended model correctly identified the single gold boundary in 99.4% of cases, compared to 39.9% for the original model.
  • Performance generally increased with the number of templates included, and models performed best when they contained templates corresponding to the "basic kind" of reduplication in the test items.

Limitations and Future Work

  • Coercion: The primary limitation identified is the coercion of certain reduplication types (e.g., left-2, right) to the full-reduplication template, particularly when bases are not observed in training. This accounts for a significant portion of errors.
  • Future Work: The authors suggest strategies like requiring the base to be longer than the reduplicant for left-2 items, or penalizing overzealous splitting of monosyllabic morphs. They also propose developing a recursive segmentation algorithm for testing that doesn't rely on pre-trained parameters.
  • Template Interaction: The interaction between templates, especially full reduplication with others, can lead to performance decreases on specific item types.
  • Future Work: Careful design and potentially more nuanced template definitions are needed to mitigate these interactions.
  • Test Set Limitations: The curated test set excluded words with long vowels and complex bases, which might limit generalizability. The comparison with fluent speakers on complex words also revealed imperfections in the reference set itself.
  • Future Work: Expanding the test set to include more complex morphological environments and refining evaluation against human judgments are important next steps.
  • Generalizability: While the Māori case study is promising, the authors suggest that the approach and insights should transfer to other Polynesian languages and potentially other languages with productive reduplication.

Summary updated

BibTeX
@inproceedings{todd2022unsupervised,
  author = {Simon Todd and Annie Huang and Jeremy Needle and Jennifer Hay and Jeanette King},
  title = {{Unsupervised morphological segmentation in a language with reduplication}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2022},
}

Loading related content...

Figure