Incorporating Complementary Annotation to a CCGbank for Improving Derivations for Japanese

Sumire Uematsu, Yusuke MiyaoIWPT 2015 · OpenReview

TL;DR

This paper tackles the problem of noisy and incomplete syntactic derivations in a Japanese CCGbank by incorporating richer annotations from the NTT treebank. By leveraging cleaner phrase structures and functional/semantic information, they improve the accuracy of Japanese sentence parsing and derivation generation, making CCG resources more robust.

What Problem Does This Paper Address?

Creating large-scale, accurate syntactic parsers often relies on automatically converting existing treebanks into formal grammars like Combinatory Categorial Grammar (CCG). However, source treebanks, especially those derived from chunk-dependencies, can be noisy. This noise manifests as errors in phrase structures and a lack of crucial linguistic information (like detailed argument structures or coordinations), which directly impacts the quality of the generated CCG derivations. For Japanese, this problem is exacerbated because chunk-dependency alone is insufficient for capturing complex structures. This paper aims to improve the Japanese CCGbank by addressing these noise and information gaps using complementary annotations.

Methods

The core of this work involves a two-step process to integrate annotations from the NTT treebank into the existing Japanese CCGbank:

  1. Reconstruction of the CCGbank: The process starts by using the cleaner phrase structures from the NTT treebank. This involves adapting the original conversion rules from chunk-dependencies to CCG derivations. A key adaptation is the introduction of a new conversion rule to handle "compound postpositions," which are multi-word expressions acting as single postpositions, a structure not explicitly handled in the original grammar.
  2. Incorporation of Additional Annotations: This step focuses on leveraging the functional tags and predicate-argument structures (PAS) from the NTT treebank.
  3. Coordination: New syntactic features (conj) and special rules are introduced to better handle noun phrase coordinations, moving beyond treating them as simple noun-noun modifications.
  4. Causer Roles: The PAS annotations, particularly for causative constructions, are used to correctly identify "causer" arguments. This involves re-analyzing causative sentences to distinguish between arguments and adjuncts, and updating categories to reflect the causer role (e.g., changing a modifier category to an argument category).

A final consistency check is performed to ensure that the modifications made to the derivations are coherent.

Datasets

  • Japanese CCGbank: The existing CCGbank for Japanese, derived from chunk-dependency resources.
  • NTT Treebank (NTB): A manually corrected version of the source treebank, containing functional tags and predicate-argument structures. This paper uses a preliminary version of the NTB containing 10,000 trees.
  • Kyoto Corpus: Used for chunk-dependencies.
  • NAIST Corpus: Used for semantic role annotations.

The 10,000 NTB sentences were divided into training, development, and test sets for evaluation.

Key Insights and Ideas

  • Complementary Annotations are Crucial: The paper strongly demonstrates that even "noisy" source treebanks can be significantly improved by overlaying richer, manually curated annotations, especially for complex linguistic phenomena like coordinations and argument structures.
  • Chunk-Dependency Limitations: It highlights that relying solely on chunk-dependency information for Japanese syntactic parsing leads to inherent limitations, particularly in accurately representing phrase structures and identifying argument roles.
  • Compound Postpositions are a Specific Challenge: Japanese compound postpositions require dedicated handling, as they function similarly to single postpositions but have more complex internal structures that need to be correctly parsed.
  • Functional Tags Unlock Deeper Analysis: Functional tags (like COORD) and semantic role information (like causer roles) are essential for disambiguating grammatical roles and improving the accuracy of syntactic derivations, moving beyond purely structural analysis.
  • 80% of Dependency Changes are "Good": A significant qualitative finding is that the majority of the changes made to the dependency relations during the reanalysis process were judged as desirable, indicating the effectiveness of the annotation integration.

Key Results

  • Improved Derivations: The process successfully modified 23% of the dependencies extracted from the original CCGbank.
  • High Quality of Changes: Manual investigation revealed that approximately 80% of these dependency changes were desirable, leading to more accurate recognition of arguments and coordinations.
  • Lexical Coverage Maintained: Lexical coverage remained high (around 98.8%), though sentential coverage dropped slightly (2-3%) due to the finer distinctions introduced by the new grammar.
  • Parsing Accuracy: While not the primary focus, training a parser on the improved CCGbank showed a slight improvement in parsing accuracy, suggesting increased structural consistency.

Limitations and Future Work

  • Limited Scope of NTB: The experiments were conducted on a preliminary version of the NTT treebank (10,000 trees), representing only 26% of the full CCGbank.
  • Causer Argument Impact: The impact of incorporating causer argument annotations was limited, partly due to the relatively small number of causer instances in the NTB subset.
  • VP Coordination: The paper notes that VP coordinations were still treated as continuous clauses, as distinguishing them from other continuous clauses based on shallow semantic information remains difficult.
  • Future Work: The authors suggest that their framework for obtaining improved CCG resources at a lower cost by integrating complementary annotations is applicable to other grammar formalisms and languages. They also aim to obtain CCG resources that allow for more precise and detailed descriptions by incorporating further annotations.

Summary updated

BibTeX
@inproceedings{uematsu2015incorporating,
  author = {Sumire Uematsu and Yusuke Miyao},
  title = {{Incorporating Complementary Annotation to a CCGbank for Improving Derivations for Japanese}},
  booktitle = {IWPT},
  year = {2015},
  url = {https://openreview.net/forum?id=dRGetp7wRIf},
}
Figure