More than Just Statistical Recurrence: Human and Machine Unsupervised Learning of Māori Word Segmentation across Morphological Processes

Ashvini Varatharaj, Simon ToddNAACL 2024

TL;DR

This paper investigates how humans learn to segment words in Māori, a language with complex morphology, and compares this to an unsupervised machine learning model (Morfessor). They find that while both humans and Morfessor excel at segmenting words formed by simple concatenation, humans outperform the model on words involving templatic morphology and other non-statistical cues, suggesting a richer learning process.

What Problem Does This Paper Address?

Understanding how humans acquire language implicitly, without explicit instruction, is a fundamental question in cognitive science. A key aspect of this is word segmentation – the ability to break down a continuous stream of speech into meaningful units. Previous research has shown that even non-native speakers with limited exposure to a language can develop implicit knowledge of its structure, including the ability to segment words. This paper specifically examines this phenomenon in Māori, a language known for its rich morphological system, which includes processes like affixation, compounding, and reduplication. The core question is whether human word segmentation is solely driven by identifying statistically recurring patterns (as modeled by algorithms like Morfessor) or if it involves more sophisticated mechanisms, such as recognizing abstract templates and other linguistic cues.

Methods

The authors compare the word segmentation performance of non-Māori-speaking New Zealanders (NMS) with that of Morfessor, an unsupervised machine learning model. Morfessor operates on the principle of statistical recurrence, aiming to find the simplest set of recurring "morphs" that can generate the training data.

The study involves two main analyses:

  1. Sensitivity to Templates: They compare segmentations of words formed by different morphological processes:
  2. Concatenative processes: Affixation and compounding, where morphemes are simply joined together.
  3. Templatic processes: Reduplication, where a part of a base is repeated according to a specific pattern. They hypothesize that Morfessor, lacking explicit template recognition, will struggle with templatic processes, while NMS might not.

  4. Sensitivity to Other Cues: They investigate if NMS utilize cues beyond statistical recurrence, such as vowel length, in word segmentation. They compare Morfessor's performance on real Māori words (which may contain these cues) versus artificially generated "pseudo-Māori" words that share statistical properties but lack these additional cues.

Performance is measured using macro-averaged precision and recall for boundary identification.

Datasets

  • Gold Standard: Segmentations provided by a fluent Māori speaker (MS) from previous work (Oh et al., 2020) and a subset of words from Panther et al. (2024).
  • Morfessor Training Data: 19,595 word types from the Te Aka Māori dictionary (Moorfield, 2011).
  • NMS Data: Word-splitting task data from non-Māori-speaking New Zealanders (Panther et al., 2024).
  • Pseudo-Māori Data: 1,000 sets of 1,292 generated words, designed to mimic the statistical properties of real Māori words.

Key Insights and Ideas

  • Humans are more than statistical learners: The most striking finding is that NMS segmentation goes beyond simple statistical recurrence. While Morfessor excels at identifying patterns, humans leverage abstract knowledge.
  • Templates matter for humans: NMS show a particular advantage in segmenting words formed by reduplication, a templatic process. This suggests humans can learn and apply abstract patterns, which Morfessor, based purely on recurrence, cannot.
  • Beyond recurrence: Other cues are utilized: The study hints that NMS also use cues like vowel length, which are not captured by Morfessor's statistical model. This is evidenced by Morfessor's poorer performance on real Māori words compared to statistically similar but artificial words.
  • Language structure influences learning: The findings underscore how the specific morphological structure of a language (like Māori) can influence the effectiveness of purely statistical learning models and highlight the need for models that can account for these structures.

Key Results

  • Monomorphemic words: Both NMS and Morfessor tend to oversegment, but NMS show slightly better performance, possibly due to sensitivity to word minimality constraints.
  • Reduplication (Templatic): NMS significantly outperform Morfessor (e.g., for total reduplication, NMS P/R: 0.95/0.97 vs. Morfessor P/R: 0.35/0.36). This strongly supports the idea that NMS learn abstract templates.
  • Affixation & Compounding (Concatenative): Morfessor performs well on these, aligning with its assumptions. However, NMS perform worse on affixation than compounding, particularly with suffixes that have many allomorphs.
  • Pseudo-Māori vs. Real Māori: Morfessor performs better on pseudo-Māori words (mean P/R: 0.84/0.96) than real Māori words (mean P/R: 0.80/0.87). This suggests that real Māori words contain cues beyond statistical recurrence that hinder Morfessor's performance.

Limitations and Future Work

  • Gold Standard Limitations: The gold standard data, derived from a single fluent speaker, might not perfectly capture all possible segmentations. Future work could involve multiple speakers to address this.
  • Morfessor vs. NMS Differences: Morfessor has perfect memory and is trained on isolated types, while NMS experience connected tokens and have memory constraints. Future research could explore how Morfessor performs when trained on data that more closely resembles NMS's experience (e.g., connected tokens, multilingual data).
  • Incorporating Templates: The study highlights the importance of templates. Future work could explore how adding template-based mechanisms to unsupervised models, like Morfessor, can improve their performance on languages with templatic morphology.
  • Broader Cues: Further investigation into other non-statistical cues (like vowel length) and their role in human and machine learning is suggested.

Summary updated

BibTeX
@inproceedings{varatharaj2024more,
  author = {Ashvini Varatharaj and Simon Todd},
  title = {{More than Just Statistical Recurrence: Human and Machine Unsupervised Learning of Māori Word Segmentation across Morphological Processes}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2024},
}
Figure