Morphological Paradigms: Computational Structure and Unsupervised Learning

Jackson L. LeeNAACL 2015

TL;DR

This thesis explores how to computationally understand and generate morphological paradigms (like verb conjugations) using unsupervised learning. It introduces novel methods for identifying word stems, measuring similarity between paradigms, and inducing entire paradigms from raw text, particularly child-directed speech. This work bridges theoretical linguistics and NLP by providing algorithms that can learn language structure without explicit supervision.

What Problem Does This Paper Address?

Morphological paradigms, such as the different forms of a verb (e.g., "walk," "walks," "walked," "walking"), are fundamental to language. Linguists and NLP researchers are interested in them because they reveal how words change predictably and how languages structure meaning. However, computationally modeling these paradigms, especially in an unsupervised manner (without pre-labeled data), is challenging. Existing methods often struggle with non-concatenative morphology (like Arabic root-and-pattern systems) or rely on assumptions about word order. Furthermore, understanding how these paradigms are acquired, particularly by children, and how to induce them from unstructured text remains an open area. This thesis tackles these challenges by proposing a three-pronged approach: identifying word stems, measuring similarity between paradigms, and inducing paradigms from raw text.

Methods

The thesis breaks down the problem into three main parts:

  • Stem Identification: This section moves beyond simple substring matching to develop language-independent algorithms for identifying the "stem" of a word within a paradigm. It explores different definitions of "maximal common material," considering linearity and contiguity, and proposes that subsequence-based approaches are more robust for handling diverse morphological types, including non-concatenative ones.
  • Paradigmatic Similarity: This part focuses on how different morphological paradigms relate to each other. It proposes a string-based hierarchical clustering algorithm to computationally characterize the similarities and differences between paradigms. This algorithm uses the non-stem material (identified in the first part) to build a distance matrix for clustering, aiming to reveal inflection classes.
  • Induction of Morphological Paradigms: This is the most ambitious part, aiming to induce entire morphological paradigms from unstructured data. Building on previous work, it proposes an incremental learning system, inspired by Linguistica, that can learn paradigms from raw text. A key innovation is its ability to incorporate the results from the stem identification and paradigmatic similarity sections, and a version is designed to process child-directed speech incrementally.

Datasets

The paper mentions the use of datasets from standard sources such as Wiktionary for paradigm tables. For the induction of morphological paradigms from unstructured data, the paper specifies using a dataset of four million word tokens from CHILDES (MacWhinney 2000) for North American English, covering an age range from a few months old to 12 years old.

Key Insights and Ideas

  • Subsequence over Substring for Stems: The paper argues that defining a stem as the maximal common subsequence (rather than substring) is more effective for handling diverse morphological phenomena, including non-concatenative morphology. This is because it prioritizes linearity over strict contiguity.
  • Clustering Reveals Linguistic Structure: Hierarchically clustering morphological paradigms can computationally reveal inflection classes, offering a more fine-grained and potentially hierarchical view of linguistic structure than traditional descriptions.
  • Incremental Learning for Child Language Acquisition: Modeling paradigm induction as an incremental process, especially using child-directed speech, is more cognitively plausible and offers insights into how children acquire language structure.
  • Bridging Theory and Computation: The thesis aims to demonstrate how theoretically informed linguistic concepts can be computationally implemented, benefiting both theoretical linguistics and practical NLP applications.

Key Results

Not discussed in the paper. The paper focuses on the proposed methods and their theoretical underpinnings rather than presenting specific quantitative results from experiments. It mentions preliminary work (Lee 2014) that suggests desirable results from clustering, and illustrates with a simplified example of English verbal paradigms.

Limitations and Future Work

  • Evaluation of Clustering: A significant open question is how to rigorously evaluate the results of clustering morphological paradigms, as standard descriptions are often not hierarchical, making a "gold standard" difficult to define. Potential evaluation methods include comparing learned classes to standard grammatical descriptions or using paradigm generation as an indirect evaluation.
  • Syntactic Information for Collapsing Paradigms: The paper suggests that inducing and combining syntactic information might be necessary for collapsing paradigms from different inflection classes without explicit annotations.
  • Incremental Learning Adaptation: While inspired by Linguistica, the proposed incremental paradigm induction system requires adaptation, particularly in its stem identification assumptions, to be language-independent.
  • Child Language Acquisition Evaluation: The evaluation of the child-directed speech induction system will be based on research on the order of acquisition of major morphological patterns in child language.

Summary updated

BibTeX
@inproceedings{lee2015morphological,
  author = {Jackson L. Lee},
  title = {{Morphological Paradigms: Computational Structure and Unsupervised Learning}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2015},
}
Figure