TL;DR
This paper introduces a novel algorithm that discovers morphologically related word pairs from raw text by combining orthographic and semantic similarity. It achieves high precision in identifying these pairs without relying on pre-existing linguistic knowledge, offering a valuable tool for linguistic analysis and computational morphology.
What Problem Does This Paper Address?
Computational linguistics has long sought to automatically identify morphological relationships between words (e.g., "run" and "running," "happy" and "happiness"). This is crucial for tasks like building stemmers, morphological analyzers, and understanding human language acquisition. Traditional approaches often require significant linguistic expertise or pre-compiled dictionaries. This paper aims to bridge this gap by developing a knowledge-free, unsupervised method that can discover these relationships directly from raw text corpora, making it applicable to new languages or domains without extensive upfront effort.
Methods
The core of the algorithm is a simple yet effective intuition: morphologically related words tend to be both orthographically and semantically similar.
- Orthographic Similarity: Measured using minimum edit distance. This quantifies the number of single-character edits (insertions, deletions, substitutions) required to transform one word into another. The authors normalize this distance by the length of the longer word to account for string length differences and then invert it (1 - normalized distance) to get a similarity score, where 1 means identical and 0 means no shared characters. Pairs with a similarity score of 0.5 or higher are considered candidates.
- Semantic Similarity: Measured using mutual information. This statistical measure quantifies how much the co-occurrence of two words deviates from what would be expected if they were independent. The authors compute this over a relatively large window (500 words) but exclude very close co-occurrences to focus on broader semantic relatedness. Only pairs co-occurring at least three times are considered.
The algorithm then combines these two scores using a weighted sum. Crucially, it does not rely on identifying specific affixes or analyzing substring distributional properties (like affix frequency), allowing it to potentially discover non-concatenative or rare morphological patterns.
Datasets
- German APA corpus: A large corpus of newswire text (over 28 million word tokens).
- English Brown corpus: A balanced corpus of English text (less than 1.2 million word tokens).
The authors also used XEROX morphological analyzers to create "reference sets" for evaluation. For English, they manually curated a subset of pairs not identified by the analyzer to improve the reference set's coverage of derivational morphology.
Key Insights and Ideas
- Orthographic and Semantic Cues are Powerful Together: The paper strongly suggests that combining simple measures of word shape (orthography) and word meaning (semantics) is a highly effective way to identify morphologically related words, even without explicit linguistic rules.
- Beyond Affixes: The method's strength lies in its ability to move beyond traditional affix-based analysis. By focusing on overall word similarity, it can uncover less common morphological processes like stem changes or circumfixation.
- Knowledge-Free is Achievable: The algorithm demonstrates that significant morphological insights can be gained from raw text without needing pre-existing dictionaries, morphological parsers, or language-specific rules.
- Precision is High, Even with Simple Methods: Despite the simplicity of the core measures, the algorithm achieves remarkably high precision, especially at the top of its ranked lists.
Key Results
- High Precision: For both German and English, the algorithm achieved precision rates above 90% for the top 1500 pairs. Even at the 5000-pair cutoff, precision remained around 50%.
- German vs. English Performance: German showed a more gradual decline in precision compared to English, which might be attributed to the larger German corpus, the richer morphological system, and potential limitations in the English reference set.
- Discovery of Diverse Patterns: Qualitative analysis revealed that the algorithm successfully captured various morphological patterns, including common affixation (e.g., -s, -ed, -ing), stem vowel changes (e.g., Mutter/Mütter), and even some non-affixal relationships like circumfixation.
Limitations and Future Work
- Precision vs. Recall: The paper focuses on precision, acknowledging that recall (finding all related pairs) is inherently difficult due to morphological productivity. The goal is to discover processes, not an exhaustive list.
- Reference Set Limitations: The evaluation relies on external analyzers, which may not capture all morphological relationships (especially derivational ones for English). Manual curation helped, but inherent biases remain.
- Future Directions:
- Iterative Refinement: Exploring iterative re-estimation of edit costs.
- Contextual Similarity: Incorporating measures of semantic context similarity.
- Morphological Transitivity: Extending the concept to find relationships like
a-bandb-cimpliesa-c. - Relaxing Semantic Thresholds: Investigating how to handle semantically opaque but morphologically complex forms.
- Improved Evaluation: Designing assessment schemes to be more directly comparable to other methods and conducting more in-depth qualitative analysis of specific morphological processes.
- Applications: Using the output as input for more sophisticated rule extractors or as a pre-processing step for linguists.
Summary updated