An Efficient Treatment of Japanese Verb Inflection for Morphological Analysis

Toru Hisamitsu, Yoshihiko NittaCOLING 1994

TL;DR

This paper tackles the challenge of efficiently analyzing Japanese verb inflections, a crucial but often overlooked aspect of Natural Language Processing (NLP). The authors propose a novel method for segmenting verbal complexes by creating new "allomorph" entries that combine stems with inflectional endings, leading to fewer lexical entries and significantly improved parsing efficiency.

What Problem Does This Paper Address?

Japanese, unlike many Western languages, doesn't explicitly mark word boundaries in its written form. This makes morphological analysis—breaking down words into their constituent parts and identifying their grammatical roles—a complex task. Verb inflection, the way verbs change form to indicate tense, mood, etc., is particularly tricky. Traditional approaches often rely on either phonological descriptions (which don't map directly to written characters) or school grammar descriptions (which can be computationally inefficient due to a large number of entries and complex transformations). The paper aims to bridge this gap by proposing a method that is both linguistically sound and computationally efficient for handling Japanese verb inflections.

Methods

The paper introduces a new lexical treatment for Japanese verb inflection. Instead of treating stems and inflectional endings as entirely separate entities or pre-generating all possible inflected forms, their method creates "allomorphs." These are essentially new lexical entries that combine a verbal stem with a specific inflectional ending (e.g., combining the stem "消s" with the negative affix "ana" to create "さな"). This approach aims to:

  • Minimize Lexical Entries: By creating these combined allomorphs, the total number of entries needed is reduced.
  • Avoid Useless Segmentation: It prevents the creation of segments that don't carry meaningful grammatical information.
  • Improve Computational Efficiency: The new segmentation and lexical structure lead to faster chart parsing.

The paper contrasts this with two existing methods: the Phonological Method (PM) and two School Grammar methods (SG-I and SG-II). PM requires converting text to romaji, which can be inefficient. SG-I pre-generates all possible inflected forms, leading to a massive dictionary. SG-II separates inflectional endings but requires extra, often meaningless, segmentation.

Datasets

The paper does not explicitly mention specific datasets for training. However, it uses a dictionary containing approximately 6000 regular verbs (class II) and 3000 regular verbs (class I) for their experiments. For quantitative comparison, they used 100 sentences from Nikkei Shinbun, totaling 5286 characters, with a dictionary of about 60000 words.

Key Insights and Ideas

  • The "Allomorph" Breakthrough: The core innovation is the creation of combined stem-allomorph entries. This elegantly bridges the gap between phonological and written forms, allowing for more direct and efficient processing.
  • Efficiency Gains are Significant: The proposed method offers substantial improvements in parsing speed (20-40% less computation) compared to existing popular methods.
  • Practical Applications Beyond Analysis: The method's efficiency and structure make it well-suited for error correction in Optical Character Readers (OCRs), a growing area of NLP application.
  • Simplicity in Construction: The reduced complexity of the lexical structure makes the morphological analyzer easier to build and maintain.

Key Results

  • Parsing Efficiency: The proposed method requires 20-40% less chart parsing computation compared to standard methods.
  • Number of Allomorphs: The number of allomorphs required is comparable to SG-II, significantly less than SG-I.
  • OCR Error Correction: Experiments showed that error correction based on the proposed method is 3.6 times faster than using the SG-I dictionary.
  • Incident Matrix Size: The proposed method reduces the size of the incident matrix, simplifying analyzer construction.

Limitations and Future Work

  • Parsing Method Dependency: While the paper uses a standard chart parser for comparison, the authors acknowledge that further experimental comparisons with other parsing methods (like the longest matching method) are needed.
  • Statistical Heuristics: The paper suggests that the proposed method is highly suitable for obtaining statistical heuristics due to its ability to intrinsically reduce free parameters, but this is presented as a direction for future exploration rather than a fully realized result.

Summary updated

BibTeX
@inproceedings{hisamitsu1994efficient,
  author = {Toru Hisamitsu and Yoshihiko Nitta},
  title = {{An Efficient Treatment of Japanese Verb Inflection for Morphological Analysis}},
  booktitle = {International Conference on Computational Linguistics},
  year = {1994},
}
Figure