OSU at SigMorphon 2022: Analogical Inflection With Rule Features

Micha Elsner, Sara CourtNAACL 2022

TL;DR

This paper introduces a novel transformer-based system for morphological inflection that augments standard transformer inputs with "rule features" derived from analogical exemplars. This approach significantly improves performance on inflection prediction tasks, especially for words with known lemmas and cells, by enabling the model to better leverage analogical reasoning.

What Problem Does This Paper Address?

Morphological inflection, the process of changing a word's form to indicate grammatical function (e.g., "walk" to "walked"), is a fundamental aspect of language. While powerful transformer models have shown promise in this area, they often struggle with exceptions and irregular forms. Traditional approaches rely on memorized word forms and analogical reasoning, where speakers infer the correct form of a new word based on similar known words and their inflected forms (e.g., "dog" : "dogs" :: "cat" : "cats"). This paper aims to bridge the gap between purely data-driven transformer models and cognitively plausible analogical reasoning, particularly when morphophonological alternations or inflection class mismatches occur between the exemplar and the target word.

Methods

The OSU system builds upon a transformer architecture but enhances it with two key innovations:

  • Analogical Exemplars: The input to the transformer is augmented with an exemplar pair (source word : target word) that demonstrates how to inflect a different word into the target grammatical cell.
  • Rule Features: To make the analogical reasoning more robust, the system generates "rule features." These features are derived by aligning the source and target words of the exemplar, and the query word and its gold output. They indicate whether changes involve prefixes, suffixes, or stem edits, and whether these changes are identical, partially matching, or disjoint between the exemplar and the target. These rule features are predicted by the transformer itself at test time.

The inference process is multistep: multiple test instances are generated with different exemplars, rule features are predicted for each, inflected forms are then predicted using these augmented inputs, and finally, a majority vote is used to select the most likely output.

Datasets

The paper mentions using datasets from SIGMORPHON 2020 for system development. For the SIGMORPHON 2022 shared task, they submitted results for Part 1 (multilingual inflection) and Part 2 (learning trajectories). For Part 1, they used the largest available datasets for each language. For Part 2, they used datasets with varying training sizes, including a 100-example dataset for Arabic.

Key Insights and Ideas

  • Analogical Reasoning is Crucial for Inflection: The success of the OSU system highlights that simply feeding raw data to a transformer isn't enough; incorporating explicit mechanisms for analogical reasoning significantly boosts performance.
  • Rule Features Bridge the Gap: The "rule features" are a clever way to guide the transformer's attention towards relevant analogical patterns, especially when direct suffix matching isn't possible. This makes the system more flexible and less prone to ignoring exemplars when they don't perfectly align.
  • Transformer Can Learn to Use Analogies: The system demonstrates that transformers can learn to interpret and utilize these rule features to make better inflection predictions, effectively learning when to attend to an exemplar.
  • Language-Specific Over-regularization: In Part 2, the system shows a tendency to over-apply the majority pattern in English, suggesting a common challenge in learning irregular forms. This is not observed in Arabic.

Key Results

  • Significant Improvement Over Baseline: The OSU system substantially improves over a baseline transformer for instances where an exemplar is available, particularly for known cells.
  • Near Challenge Winner Performance: For known cells, the system's performance is close to the challenge winner (CLUZH), with differences being statistically significant but relatively small.
  • Part 1 Performance: The system achieved competitive results on Task 0, Part 1, outperforming the neural baseline by almost 10% absolute when the cell was known. It performed worse than the baseline on some languages (e.g., Arabic, Korean), with no obvious typological pattern.
  • Part 2 Learning Curves: For Arabic (and qualitatively similar for English and German), the system shows rapid performance gains between 100 and 300 training items, with diminishing returns around 600 items.

Limitations and Future Work

  • Inability to Predict Fillers: A significant limitation is the system's inability to predict fillers for cells where no training example is given. This is particularly problematic for languages with very large paradigms.
  • Future Work: The authors suggest that modeling relationships between cells using multiple input forms to predict a target could address the "filler" problem. They also believe that their analogical approach is cognitively plausible and a valuable addition for building more robust and generalizable inflection systems.

Summary updated

BibTeX
@inproceedings{elsner2022osu,
  author = {Micha Elsner and Sara Court},
  title = {{OSU at SigMorphon 2022: Analogical Inflection With Rule Features}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2022},
}
Figure