TL;DR
This paper presents JB132's submission to the SIGMORPHON 2022 Shared Task 3 on morphological segmentation. The authors developed a modular probabilistic framework using Expectation-Maximization to segment words into their constituent morphemes, but found their approach yielded poor results, prompting a detailed error analysis.
What Problem Does This Paper Address?
Morphological segmentation, the task of breaking down words into their smallest meaningful units (morphemes like prefixes, roots, and suffixes), is crucial for understanding language structure and for downstream NLP tasks. However, accurately segmenting words, especially in morphologically rich languages, is challenging. This paper addresses the SIGMORPHON 2022 Shared Task 3, which specifically focuses on this segmentation problem, aiming to develop models that can effectively identify morpheme boundaries.
Methods
The core of JB132's submission is a modular probabilistic framework built around the Expectation-Maximization (EM) algorithm. This framework allows for rapid testing of various model architectures. Their final architecture consists of two main components:
- Word-Morpheme Generator: This part models the high-level structure of words as a sequence of morphemes. It uses a Hidden Markov Model (HMM) with three states (prefix, root, suffix) and employs dictionaries for known morphemes, falling back to a character-level unigram model for unknown ones.
- Morpheme-Morph Model: This component handles the realization of morphemes into actual word segments (morphs). It assigns probabilities to each morpheme generating specific morphs.
Both components are trained separately using EM. The training process involves a recursive approach to find the most probable generation of a word and then updating model parameters based on these generated structures.
Datasets
The training data for the SIGMORPHON 2022 Shared Task 3 spanned nine languages: Czech, English, French, Hungarian, Spanish, Italian, Latin, Russian, and Mongolian. The datasets contained tens of thousands to hundreds of thousands of training samples. The paper notes that the input data varied in complexity, with Czech words segmented to "morphs" and other languages to "morphemes," sometimes including root changes and null morphs.
Key Insights and Ideas
- Modularity for Rapid Prototyping: The authors emphasize the benefit of their modular framework, which allowed them to quickly test different probabilistic models and architectures.
- EM's Suitability for Segmentation: The choice of EM is motivated by its tendency to simplify loss functions, leading to faster computation, a common goal in probabilistic modeling for segmentation.
- The Challenge of Root Changes: A significant insight from their error analysis is the difficulty in capturing root changes, where the underlying root morpheme transforms in its surface realization (e.g., "emulations" from "emulate").
- Context is King (or Missing): The lack of contextual information in their model is identified as a major limitation, preventing it from leveraging joint probabilities of morphemes and leading to errors in morph selection.
- The "Empty Morph" Problem: Probabilistic models struggle with morphemes that don't have a direct surface realization (empty morphs), highlighting the need for models that can infer these implicitly.
- Czech Data Nuances: The segmentation of Czech data into "morphs" rather than "morphemes" and the prevalence of single-letter morphs presented unique challenges, leading to under-segmentation when these were removed.
Key Results
The JB132 submission achieved relatively poor results across most languages. The F-scores reported are:
- Czech: 64.65
- English: 65.43
- French: 46.20
- Hungarian: 72.64
- Italian: 33.44
- Latin: 91.39
- Mongolian: 57.82
- Russian: 50.55
- Spanish: 43.39
The model performed best on Latin, which the authors note was segmented to morphs, suggesting a potential alignment with their model's capabilities. However, on other languages, the results were not competitive with other approaches.
Limitations and Future Work
The authors acknowledge several limitations:
- Inability to Capture Root Changes: The model struggles with transformations of root morphemes.
- Lack of Context: The model does not consider surrounding letters or joint probabilities of morphemes, leading to errors.
- Handling Empty Morphs: The probabilistic model cannot generate morphemes from nothing.
- Under-segmentation: Particularly on Czech data, the model missed boundaries, partly due to the removal of single-letter morphemes.
- Root Boundary Detection: The model has trouble identifying the exact start and end of root morphemes.
Future work suggested includes incorporating more contextual information and developing a mechanism to bridge the gap between root morphs in words and root morphemes in dictionaries. They also suggest exploring models that group short morphs or incorporate joint probabilities to address issues like under-segmentation.
Summary updated