Abstract
Morphological segmentation is beneficial for several natural language processing tasks dealing with large vocabularies. Unsupervised methods for morphological segmentation are essential for handling a diverse set of languages, including low-resource languages. Eskander et al. (2016) introduced LIMS, a system that uses Adaptor Grammars (AGs) for this task. However, while LIMS worked best on average and outperforms other state-of-the-art unsupervised morphological segmentation approaches, it did not provide the optimal AG configuration for five out of the six languages. We propose two language-independent classifiers that enable the selection of the optimal or nearly-optimal configuration for the morphological segmentation of unseen languages.
Introduction
Morphological segmentation—the process of decomposing words into their constituent morphemes (e.g., re-play-ing-s)—is a critical preprocessing step in Natural Language Processing (NLP). This task is particularly vital for morphologically rich languages where the vocabulary size can explode due to complex word formations. While supervised segmentation models achieve high performance, they rely on extensive annotated corpora that are unavailable for the vast majority of the world's languages.
Unsupervised approaches, such as those utilizing Adaptor Grammars (AGs), offer a promising alternative by learning word structures directly from raw text. However, a significant challenge remains: linguistic diversity. A grammar configuration optimized for a suffixing language like Turkish may perform poorly on a prefixing or templatic language. This paper addresses the "one-size-fits-all" limitation by introducing a method to automatically select the optimal grammar configuration for any given language.
Methodology: Adaptive Grammar Selection
The proposed approach builds upon the LIMS system, which uses Adaptor Grammars—nonparametric Bayesian models that generalize probabilistic context-free grammars. The researchers focus on selecting the best configuration across two primary dimensions: the learning setup (Standard vs. Cascaded) and the grammar type.
To automate this selection, a two-stage classification pipeline is employed:
- Fast-Probing Feature Generation: Instead of running the full unsupervised learning process (typically 500 iterations), the system performs a "light" run of only 50 iterations. From this preliminary output, 14 morphological features are extracted, such as the average number of suffixes per word and the count of distinct complex affixes.
- Language Classification: These features serve as input for a classifier (K-Nearest Neighbors, Naive Bayes, or Random Forest) trained to predict the configuration that will maximize the EMMA F-measure for the specific language.
Experimental Setup
The study evaluates the methodology across a diverse set of languages:
- Development Set: English, German, Finnish, Turkish, Estonian (from Morpho Challenge), and Zulu (from the Ukwabelana corpus).
- Unseen Test Language: Arabic (from the Penn Arabic Treebank), chosen for its distinct non-concatenative morphology to test the model's generalizability.
- Data: Each language utilized a training corpus of approximately 50,000 words.
Results and Discussion
The K-Nearest Neighbors (KNN) classifier demonstrated exceptional performance, achieving 100% accuracy in leave-one-out cross-validation across the development languages. It correctly identified that different languages require distinct grammar setups to achieve peak performance.
Crucially, the system successfully generalized to the unseen test language, Arabic, identifying the "Standard PrStSu+SM" configuration as optimal. This resulted in an EMMA F-score of 0.701, significantly outperforming the previous LIMS "best-on-average" baseline (0.682) and other standard tools like Morfessor (0.678).
Key Insights
- Suffixes as Indicators: Feature analysis revealed that suffix-related statistics are the most potent predictors of a language's morphological type.
- Efficiency of Fast-Probing: The results confirm that only 10% of the standard optimization iterations are necessary to capture the linguistic signal required for accurate model selection.
- Avoiding the "Average" Trap: The previous state-of-the-art was only optimal for one language (Zulu) in the development set, highlighting the necessity of language-specific tailoring.
Conclusion and Future Work
By automatically tailoring Adaptor Grammar configurations, this method achieves substantial error reductions—17.1% over Morfessor and 6.3% over the LIMS baseline. Future research will focus on expanding the training set to include more diverse language families and evaluating the impact of these improved segmentations on downstream NLP tasks like Machine Translation.
Summary updated