Morphological reinflection with weighted finite-state transducers

Alice Kwak, Michael Hammond, Cheyenne WingACL 2023

TL;DR

This paper explores non-neural approaches to morphological reinflection, focusing on how to model word transformations using weighted finite-state transducers (WFSTs). The authors investigate the impact of linguistic features like word frequency and length, ultimately proposing an ensemble system that combines their best-performing models.

What Problem Does This Paper Address?

The paper tackles the challenge of morphological reinflection, which is the task of generating different forms of a word based on grammatical rules (e.g., "walk" -> "walked," "walking"). Specifically, the SIGMORPHON 2023 Shared Task aims to model this for typologically diverse languages, meaning languages with very different morphological structures. The core problem is learning these complex relationships between a word's base form (lemma), its grammatical properties (morphosyntactic description), and its inflected form. Traditional approaches often struggle with the sheer variety of morphological phenomena across languages, especially nonconcatenative morphology (where changes happen within the word stem, not just by adding prefixes/suffixes).

Methods

The authors explore several non-neural methods, building upon a provided baseline:

  • Non-neural Baseline: This system, inspired by prior work, uses an alignment algorithm to hypothesize potential prefix and suffix rules based on lemma-surface form pairs. During inference, it selects the best prefix and suffix rules based on length and frequency.
  • Baseline Improvements: They enhance the baseline by:
  • Modifying the input for prefix rule extraction to improve handling of cases where lemma and word forms don't share an obvious root.
  • Asymmetrically applying criteria for choosing the best prefix and suffix rules, ensuring both length and frequency are considered for prefixes as they are for suffixes.
  • Weighted Finite-State Transducers (WFSTs): This is their primary novel approach. They build WFSTs by:
  • Using edit distance to find the optimal alignment between a lemma and its inflected form.
  • Replacing identical spans in the alignment with variables that carry weights penalizing mismatches.
  • Implementing these WFSTs using the pyfoma system.
  • They experimented with different weighting strategies, including penalizing shorter, less frequent rules and adjusting weights based on output form frequency.
  • Ensemble System: A final system that combines the outputs of their improved baseline and WFST system, selecting the best result based on development data performance.

Datasets

The paper uses data from the SIGMORPHON 2023 Shared Task, which comprises 10,000 examples of inflected forms across 27 languages. The data includes the lemma, morphosyntactic category, and the inflected form. Morphosyntactic information is provided in the Unimorph format.

Key Insights and Ideas

  • Frequency and Length Matter: Similar to the baseline, incorporating word frequency and length into rule selection significantly improved performance, highlighting their importance even in non-neural models.
  • WFSTs for Morphological Modeling: The authors demonstrate that WFSTs can be a viable, albeit complex, tool for modeling morphological inflection. Their approach of using edit distance to build WFSTs is relatively simple and interpretable.
  • Challenges with Nonconcatenative Morphology: Despite initial hypotheses, their WFST approach did not inherently capture nonconcatenative morphology better than concatenative cases. This suggests that modeling internal stem changes requires more sophisticated WFST structures or different approaches.
  • Interpretability of WFST Weights: A key strength of their WFST system is its interpretability. The weights can be manipulated to test the effect of different linguistic factors (frequency, length, etc.) on performance.
  • Ensembling for Robustness: Combining multiple models, even if individually imperfect, can lead to better overall performance, especially across diverse languages.

Key Results

  • The improved non-neural baseline achieved 71.7% accuracy, a modest but notable improvement over the original baseline.
  • The WFST-based system (Submission 3) performed less well overall at 56.1%, but showed specific improvements for languages like Armenian, Italian, and Japanese.
  • The ensemble system (Submission 4) achieved the best overall performance at 72.4%.
  • Adjusting WFST weights based on output form frequency provided a 5% boost in development data performance.

Limitations and Future Work

  • WFST Performance: The WFST system did not perform as well as the non-neural baseline overall, particularly for languages with complex morphology.
  • Compiled WFSTs: The authors suggest moving to a compiled WFST system to more thoroughly test "individual variable" models and potentially reduce computational costs.
  • Training WFST Weights: Currently, WFST weights are set naively. Training these weights, rather than setting them in advance, is a promising direction.
  • Variable Weighting by Locality: Investigating whether mismatched variables have more impact when they are closer to the point of change could improve WFST models.
  • Granularity of Variables: Adjusting the granularity of variables (e.g., characters vs. spans) based on position might be beneficial.
  • Combining WFSTs: Merging individual lemma WFSTs into a single, larger WFST could be more efficient.

Summary updated

BibTeX
@inproceedings{kwak2023morphological,
  author = {Alice Kwak and Michael Hammond and Cheyenne Wing},
  title = {{Morphological reinflection with weighted finite-state transducers}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2023},
}
Figure