Chinese Dependency Parsing with Large Scale Automatically Constructed Case Structures

Kun Yu, Daisuke Kawahara, Sadao KurohashiCOLING 2008

TL;DR

This paper introduces a novel approach to improve Chinese dependency parsing by automatically constructing and utilizing large-scale "case structures." These structures capture richer syntactic information than traditional methods, leading to a significant boost in parsing accuracy.

What Problem Does This Paper Address?

Dependency parsing aims to represent the grammatical structure of a sentence by identifying relationships between words (head-modifier). While effective for many languages, Chinese dependency parsing faces challenges due to its rich morphology and lack of explicit grammatical markers like prepositions or case endings. Existing methods often struggle to capture the nuanced relationships between words, particularly when dealing with complex sentence structures. This paper addresses this gap by proposing a new type of "case structure" that goes beyond simple lexical dependencies to incorporate parsing history, thereby improving the accuracy of Chinese dependency parsers.

Methods

The core innovation lies in the proposed "Chinese case structure," which has two key characteristics:

  1. Relaxed Predicate Definition: Unlike traditional case structures that focus on verbs as predicates, this approach considers any word that acts as a head in a dependency relation as a potential predicate.
  2. History-Based Marking: Instead of relying on semantic roles, these case structures are marked by the sequence of neighboring modifiers attached to a head. This "case pattern" effectively remembers the parsing history of a head node, helping to disambiguate complex relationships.

These case structures are automatically constructed using a two-pronged approach:

  • Small Tagged Corpus: A portion of the Penn Chinese Treebank is used for initial learning.
  • Large Raw Corpus: A massive dataset (Chinese Gigaword) is leveraged to build a much larger set of case structures.

A deterministic Chinese dependency parser is used to process the raw corpus, and a Head-Modifier (HM) classifier, based on SVM, is employed to ensure the quality of extracted head-modifier pairs. The parser itself uses a CKY algorithm to decode the dependency tree, incorporating the probabilities derived from the constructed case structures.

Datasets

  • Penn Chinese Treebank 5.1: Used as the tagged corpus for training and evaluation.
  • 9,684 sentences for training case structures.
  • 346 sentences for testing.
  • 334 sentences for development.
  • Chinese Gigaword: A large raw corpus (7,338,028 sentences) used for automatically constructing case structures.

Key Insights and Ideas

  • Parsing History Matters: The paper strongly suggests that remembering the sequence of modifiers attached to a head (parsing history) is crucial for accurate dependency parsing, especially in languages like Chinese.
  • Beyond Verbs as Predicates: Broadening the definition of a "predicate" to include any head word significantly expands the applicability of case structures.
  • Leveraging Raw Text: The successful construction of a large-scale case structure resource from raw text demonstrates the power of unsupervised or semi-supervised learning for NLP tasks.
  • Combined Corpus Approach: Using both tagged and raw corpora for case structure construction yields better results than using either alone, highlighting the complementary strengths of each.

Key Results

The proposed Chinese dependency parser, utilizing the automatically constructed case structures, achieved an unlabeled attachment score (UAS) of 87.26% on the Penn Chinese Treebank. This represents a significant improvement of 2.13% over the baseline parser that did not use case structures. The paper also shows that using only case elements (lexical preferences) improved accuracy by 1.30%, indicating that both case elements and case patterns contribute to the overall performance, with case elements having a slightly larger impact.

Limitations and Future Work

  • Dependency on Parser Quality: The accuracy of the automatically constructed case structures is dependent on the quality of the parser used on the raw corpus. Errors in the initial parsing can lead to incorrect case structures.
  • Case Pattern Representation: The current representation of case patterns might be too strict, leading to many patterns existing only in one corpus. The authors suggest modifying this representation for greater flexibility.
  • Parse Selection: Enhancing "good parse selection" on the raw corpus could improve the quality of extracted head-modifier pairs and address issues with incorrect pairs.
  • Integrating POS Tagging: The paper notes that incorrect POS tagging can significantly affect parsing accuracy and suggests integrating POS tagging into the parsing model as a future direction.
  • Language Adaptability: The authors believe the core idea of using history-based case structures could be adapted to other languages without requiring language-specific knowledge.

Summary updated

BibTeX
@inproceedings{yu2008chinese,
  author = {Kun Yu and Daisuke Kawahara and Sadao Kurohashi},
  title = {{Chinese Dependency Parsing with Large Scale Automatically Constructed Case Structures}},
  booktitle = {International Conference on Computational Linguistics},
  year = {2008},
}

Loading related content...

Figure