Were We There Already? Applying Minimal Generalization to the SIGMORPHON-UniMorph Shared Task on Cognitively Plausible Morphological Inflection

Colin Wilson, Jane S.y. LiACL 2021

TL;DR

This paper explores how well a computational model based on "minimal generalization" can predict human-like morphological inflections. By applying this learning principle to German, English, and Dutch, the model achieved competitive results on a shared task, suggesting that simple, cognitively plausible learning mechanisms can explain complex linguistic phenomena.

What Problem Does This Paper Address?

How do humans learn to generate new words with correct grammatical endings (morphological inflection)? Specifically, how do they generalize from a limited set of examples to produce plausible forms for novel words (like the famous "wug test")? Traditional computational models often rely on complex mechanisms or vast amounts of data. This paper investigates whether a simpler, more cognitively grounded principle – minimal generalization – can account for human-like inflectional behavior. Minimal generalization suggests that learners form rules by finding the smallest possible changes that explain observed word pairs, avoiding over-generalization. The challenge is to see if this principle, when applied computationally, can match human performance on a standardized task.

Methods

The core of the paper is the application of a computational model based on minimal generalization, as originally proposed by Albright and Hayes. This model works by:

  • Learning Base Rules: For each known word pair (e.g., a verb stem and its past tense form), it identifies the longest common prefix and suffix, and then infers the minimal change (addition, deletion, or substitution of segments) needed to transform one wordform into the other. This results in a lexeme-specific rule.
  • Recursive Minimal Generalization: The model then iteratively applies the minimal generalization operation to pairs of existing rules. This operation combines two rules that perform the same change by finding commonalities in their contexts (left and right) and generalizing them. This process is repeated until no new, more general rules can be formed.
  • Pruning and Scoring: To manage the potentially large number of learned rules, the authors implement methods to prune rules based on their generality and accuracy (confidence score).
  • Prediction: The remaining rules are then used to predict inflected forms for novel "wug" words.

The authors also developed a command-line interface for the original GUI-based model and preprocessed the input data, including handling phonological features and segment representations.

Datasets

The study uses data from the SIGMORPHON-UniMorph 2021 Shared Task. This task involved predicting human wug-test ratings for morphological inflections in three languages:

  • German: Past participle formation.
  • English: Past tense formation.
  • Dutch: Past tense formation.

The paper does not introduce new benchmark datasets but rather applies the model to existing ones for evaluation.

Key Insights and Ideas

  • Minimal Generalization as a Cognitive Principle: The study strongly supports the idea that minimal generalization is a powerful and cognitively plausible mechanism for learning morphological rules. The model's competitive performance suggests that humans might not need overly complex learning strategies.
  • Simplicity Can Be Powerful: By stripping down the model to focus purely on minimal generalization (and ablating other potential mechanisms like learning phonological rules or impugnment), the authors demonstrate its effectiveness. This highlights the importance of parsimony in cognitive models.
  • The Challenge of Over- vs. Under-generalization: The paper grapples with the balance between creating rules that are too specific (under-generalization) and too broad (over-generalization). The recursive application of minimal generalization aims to find a sweet spot, but the authors also explore methods to construct wug items that can specifically test for under-generalization.
  • Cross-linguistic Applicability: The model's success across German, English, and Dutch suggests that the core principle of minimal generalization is applicable to different morphological systems.

Key Results

The model achieved competitive results on the SIGMORPHON-UniMorph 2021 Shared Task. Specifically:

  • The model achieved the best AIC (Akaike Information Criterion) for the German and English test data, and the best overall AIC summed across all languages. Lower AIC values indicate a better match between model predictions and average human wug-test ratings.
  • The number of learned rules, even after pruning, was substantial, ranging from thousands to tens of thousands per language.
  • The authors found that a specific scoring method (confidence) worked best for German and English, while another (score10) was better for Dutch.
  • A penalty was added for English past tense predictions to account for human participants' potential aversion to "double past" inflections, which improved performance.

Limitations and Future Work

  • Limitations:
    • The model learns rules for individual morphological relations (e.g., past tense) rather than entire morphological systems jointly.
  • The current version does not incorporate phonological rule learning or "impugnment" (a mechanism to limit generalization).
  • The model cannot learn sensible rules for circumfixes.
  • The original model's GUI interface was a barrier, though the authors created a command-line interface.
  • Future Work:
    • Extensions: Incorporating phonological rule learning, impugnment, and "doppelgänger" rules (cross-context generalization).
  • Empirical Testing: Developing new wug-test stimuli to specifically probe for under-generalization and compare the model's predictions with human behavior in more nuanced ways.
  • Investigating Lexeme Frequency: Exploring how incorporating lexeme frequencies (as in the original Albright and Hayes implementation) could influence phonological rule learning.
  • Alternative Models: The authors acknowledge that if human data doesn't align with predictions, it might suggest alternative models, such as those relying more on detailed exemplars.

Summary updated

BibTeX
@inproceedings{wilson2021were,
  author = {Colin Wilson and Jane S.Y. Li},
  title = {{Were We There Already? Applying Minimal Generalization to the SIGMORPHON-UniMorph Shared Task on Cognitively Plausible Morphological Inflection}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2021},
}
Figure