Abstract
This paper details the submission by researchers from Columbia University and NYU Abu Dhabi to the SIGMORPHON 2016 Morphological Reinflection shared task. The authors adapted a supervised paradigm completion system by integrating a custom automated word segmentation phase. This pipeline achieved significant performance gains, including a 30% absolute accuracy boost over the baseline in Turkish and rankings as high as 3rd place in specific task tracks. The study highlights the critical role of the data-to-feature ratio in morphological modeling success.
Introduction
Morphological reinflection is the task of generating a target word form given a lemma (dictionary form) and a set of target grammatical features, such as tense, number, or case. This task is inherently difficult due to the vast diversity of human languages, which may utilize prefixes, suffixes, or internal vowel changes (apophony) to express these categories.
While many state-of-the-art systems require "gold standard" segmented data to learn these patterns, the SIGMORPHON 2016 shared task provided only raw strings. To address this, the authors developed a system that "sandwiches" core paradigm completion logic between new automated preprocessing and post-processing layers.
Methodology
The system follows a three-stage pipeline designed to handle raw, unsegmented data:
- Automated Word Segmentation: The authors developed a three-step probabilistic approach to estimate the likelihood of prefix-stem-suffix combinations based on part-of-speech (POS) and features. By clustering words by lemma and choosing segmentations that minimize the number of unique stems per cluster, the system enforces stem consistency.
- Paradigm Completion: Using the segmented data, the system builds Inflectional Classes (ICs). Stems are abstracted into templates by identifying "root letters" and "pattern letters" (letters that change, such as the i in sing/sang/sung), allowing the system to generalize patterns to unseen words.
- Reinflection: To generate a new form, the system matches the input against its learned ICs. If the lemma was seen during training, it retrieves the form directly; otherwise, it uses the IC as a morphological analyzer to predict the correct transformation.
Experimental Results
The system was evaluated on eight diverse languages: Arabic, Georgian, German, Hungarian, Maltese, Navajo, Spanish, and Turkish. As shown in the performance data, the system significantly outperformed the shared task baseline in six out of eight languages.
The most dramatic improvement occurred in Turkish, where accuracy rose from a 54% baseline to 84%. In Navajo, the system achieved 84% accuracy compared to a 60% baseline. Conversely, the system struggled with Maltese (29%) and Arabic (59%), where complex orthography and high feature counts presented significant challenges.
Key Insights and Analysis
- The "Data-to-Feature" Ratio: A primary finding was the strong correlation (0.94) between the ratio of training examples to morphological features (log(E/F)) and system success. Languages with "rich" morphology require substantially more data to model accurately.
- Segmentation as a Bottleneck: Most errors were traced back to the initial segmentation phase. Incorrect stem identification creates a "cascading error" that leads the system to select the wrong inflectional paradigm.
- Orthography vs. Phonology: The system faced difficulties with languages like Arabic, where the writing system (orthography) hides phonetic changes. For example, "sun letters" cause the definite article al- to assimilate into the following consonant, a rule difficult to capture through character-level analysis alone.
Conclusion and Future Work
The authors identified three primary error categories: stemming errors, paradigm selection errors, and allomorphy issues (failing to handle multiple forms of a single affix). Future research will focus on joint segmentation and paradigm completion to solve these problems simultaneously rather than linearly. Additionally, the team intends to explore deep learning models to better detect allomorphy and handle complex phonological rewrites.
Summary updated