A Hybrid Japanese Parser with Hand-crafted Grammar and Statistics

Hiroshi Kanayama, Kentaro Torisawa, Yutaka Mitsuishi, Jun’ichi TsujiiCOLING 2000

TL;DR

This paper introduces a novel hybrid parsing method for Japanese that combines a hand-crafted grammar with statistical techniques. By leveraging alternative parse trees generated by the grammar to inform statistical likelihood estimation, they developed a "Triplet/Quadruplet Model" that significantly improves shallow-level dependency analysis accuracy.

What Problem Does This Paper Address?

Syntactic parsing, especially for morphologically rich and relatively free word-order languages like Japanese, is a challenging task. While purely statistical methods have achieved impressive results in shallow parsing, they often lack the precision and semantic representation capabilities of hand-crafted grammars. Conversely, grammar-based systems can be brittle and struggle with the ambiguity inherent in natural language. The authors aim to bridge this gap by creating a hybrid system that harnesses the strengths of both approaches, using a grammar to guide and enrich statistical models for more accurate shallow parsing.

Methods

The core innovation is the "Triplet/Quadruplet Model," a statistical approach to estimating the likelihood of a parse tree. Unlike previous models that relied solely on attributes of the modifier and modifiee, this model incorporates information from alternative parse trees generated by an HPSG-based grammar (SLUNG). Specifically, when calculating the probability of a dependency between two words (n and h), the model considers the attributes of the modifier's non-head daughter and its alternative modification candidates (other potential words it could modify).

This approach leads to two key advantages:

  1. A New Distance Metric: Instead of an absolute distance between words, the model uses the relative position of a candidate modifiee among the grammatically licensed alternatives. This helps to ignore "noise" from intervening words that don't affect the core dependency.
  2. Treating Alternative Trees: By considering multiple potential parse structures simultaneously, the model can better disambiguate dependencies, especially when grammatical rules allow for multiple interpretations.

To manage data sparsity, the number of modification candidates considered is restricted to at most three. The system uses a maximum entropy estimation framework for learning.

Datasets

  • EDR Japanese Corpus: Used for both training and evaluation. This corpus contains 208,157 sentences from newspapers and magazines.
  • 192,778 sentences for training.
  • 6,744 sentences for pre-analysis.
  • 3,372 sentences for testing.
  • Kyoto University Corpus: Used for comparative evaluation with another model. The paper applied their method to 1,246 sentences from this corpus.

Key Insights and Ideas

  • Grammars as Feature Extractors: The paper demonstrates that hand-crafted grammars, even underspecified ones, can be invaluable for extracting rich features that are difficult to obtain otherwise, thereby improving statistical models.
  • Context is Key, But What Context? The authors highlight that considering only local context (modifier/modifiee attributes) is insufficient. Their work shows the power of considering grammatically licensed alternatives as a form of contextual information.
  • Relative Position Over Absolute Distance: The proposed relative positioning metric for candidates proves more robust than simple word counts, as it can filter out irrelevant intervening words.
  • Hybridization Pays Off: The results strongly suggest that combining symbolic grammar knowledge with statistical learning is a fruitful path for improving parsing accuracy, particularly for shallow analysis.

Key Results

The hybrid parser achieved a bunsetsu accuracy of 88.6% on the EDR annotated corpus, outperforming other purely statistical parsing methods. This represents a significant improvement, especially considering the complexity of Japanese dependency analysis. The paper also shows that the grammar's contribution (W/O Grammar vs. W/O Restriction) and the Triplet/Quadruplet model itself (Pair vs. Triplet/Quadruplet) both contribute to this accuracy gain.

Limitations and Future Work

  • Kyoto Corpus Discrepancy: While achieving high accuracy on the EDR corpus, the model's performance on the Kyoto Corpus was worse than a comparable model. The authors attribute this to differences in annotation schemes, morphological analyzers (JUMAN vs. others), and the grammar (SLUNG) being tailored for EDR.
  • Semantic Representations: The current work focuses on shallow syntactic analysis. Future work aims to extend the SLUNG grammar to produce semantic representations, moving towards more intelligent natural language processing systems.
  • Speed: While recent advances have made HPSG parsing faster, the authors acknowledge that grammar-based systems can be slower than purely statistical ones. They report an average parsing time of 0.5 seconds per sentence for the EDR corpus.

Summary updated

BibTeX
@inproceedings{kanayama2000hybrid,
  author = {Hiroshi Kanayama and Kentaro Torisawa and Yutaka Mitsuishi and Jun’ichi Tsujii},
  title = {{A Hybrid Japanese Parser with Hand-crafted Grammar and Statistics}},
  booktitle = {International Conference on Computational Linguistics},
  year = {2000},
}
Figure