Knowledge-Free Induction of Inflectional Morphologies

Patrick Schone, Daniel JurafskyNAACL 2001 · OpenReview

Abstract

We propose an algorithm to automatically induce the morphology of inflectional languages using only text corpora and no human input. Our algorithm combines cues from orthography, semantics, and syntactic distributions to induce morphological relationships in German, Dutch, and English. Using CELEX as a gold standard for evaluation, we show our algorithm to be an improvement over any knowledge-free algorithm yet proposed.

Introduction

Morphological analysis—understanding how words are built from smaller units like stems and affixes—is a cornerstone of Natural Language Processing (NLP). Traditionally, building these analyzers required linguists to manually write thousands of rules, a labor-intensive process that doesn't scale well to new languages. While previous machine-learning attempts tried to automate this, they usually required human-provided root words or suffix tables. This paper tackles the "knowledge-free" challenge: inducing the full morphological structure of a language (including prefixes, suffixes, and complex "circumfixes") without any human-provided rules or dictionaries.

Methodology

The authors propose a multi-stage pipeline that mimics how a human might deduce patterns from a massive pile of text. The process integrates spelling patterns, word meanings, and local sentence structure to identify potential morphological variants.

The 7-step pipeline for morphological induction.
  1. Candidate Generation (Tries): The system uses character trees (tries) to find common word beginnings and endings, identifying "pseudo-prefixes" and "pseudo-suffixes" based on frequency.
  2. Semantic Filtering (LSA): To avoid deceptive spelling (e.g., "car" vs. "cares"), Latent Semantic Analysis (LSA) creates "meaning vectors." Relationships are discarded if words share a stem but have divergent meanings.
  3. Orthographic Boosting: Affix frequency and Minimum Edit Distance are incorporated. Highly frequent rules (like adding "-s") receive higher probabilities.
  4. Local Syntactic Context: The algorithm examines immediate neighbors. For example, if "these" often precedes word A and "this" precedes word B, it signals that A is likely a plural version of B.
  5. Transitive Closure: A graph-based approach fills in "missing links." If "abuse" relates to "abused," and "abused" relates to "abusive," the system infers a relationship between "abuse" and "abusive."

Key Insights: The Morphological Web

A central innovation is the "Noisy Or" integration. While semantics is a powerful filter, it can fail for "semantically opaque" words (like "reusability" vs "use"). By combining semantic, orthographic, and syntactic signals, the system becomes more robust than any single-signal approach.

Morphological relationships and confidence scores for the root 'abuse'.

Furthermore, the system treats capitalization as a type of prefix, naturally grouping "Apple" and "apple" without special pre-processing. The use of transitive closure allows the system to build a comprehensive web of relationships, significantly reducing missed connections (deletions) in the final morphological graph.

Evaluation and Results

The algorithm was trained on large-scale corpora: English (6.7M words), German (2.3M words), and Dutch (6.7M words). Results were compared against the CELEX lexical database gold standard.

The integrated approach showed a steady climb in performance as each new feature was added:

  • English Performance: Achieved an 88.1% F-score, outperforming previous state-of-the-art methods like Goldsmith’s Linguistica (81.8%).
  • Cross-Lingual Success: The system performed even better on German (92.3%) and Dutch (85.8%).
  • Error Reduction: Combining orthography, syntax, and transitivity resulted in a nearly 20% relative reduction in error compared to using semantics alone.

Conclusion

While the system is excellent at regular morphology, it still struggles with highly irregular forms (e.g., "go" vs. "went") and requires sufficient word frequency for reliable semantic vectors. Future work suggests coupling this induction with specialized techniques for irregular forms to create a universal language learner.

Summary updated

Abstract

We propose an algorithm to automatically induce the morphology of inflectional languages using only text corpora and no human input. Our algorithm combines cues from orthography, semantics, and syntactic distributions to induce morphological relationships in German, Dutch, and English. Using CELEX as a gold standard for evaluation, we show our algorithm to be an improvement over any knowledge-free algorithm yet proposed.
BibTeX
@inproceedings{schone2001knowledge,
  author = {Patrick Schone and Daniel Jurafsky},
  title = {{Knowledge-Free Induction of Inflectional Morphologies}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2001},
  url = {https://openreview.net/forum?id=H1-Ujz-dZH},
}
Figure