A Lightweight Semantic Chunker Based on Tagging

Kadri HaciogluNAACL 2004

TL;DR

This paper introduces a novel "phrase-by-phrase" semantic chunking approach that represents sentence meaning using shallow, non-overlapping lexicalized trees. This method achieves better performance than existing "word-by-word" techniques, offering a more portable, faster, and lightweight solution for semantic role labeling.

What Problem Does This Paper Address?

Extracting semantic representations from text is crucial for many Natural Language Processing (NLP) tasks like question answering and information extraction. Traditionally, this involves defining frames with slots to capture domain-specific knowledge, which is costly and difficult to adapt to new domains. Domain-independent approaches, like predicate-argument structure, offer a more general solution. However, existing methods often rely on computationally expensive full syntactic parsers and complex treebank annotations. This paper aims to bridge this gap by developing a fast, accurate, and portable semantic chunker that avoids deep syntactic parsing and complex annotation, making semantic analysis more accessible.

Methods

The core innovation is a "phrase-by-phrase" (P-by-P) semantic chunking strategy. Instead of processing words individually, it operates on shallow, non-overlapping, lexicalized trees at lexical, syntactic, and semantic levels. The process is framed as a sequence of tagging actions, moving from bottom-up.

  1. POS Tagging: Words are first tagged with their Part-Of-Speech (POS).
  2. Syntactic Chunking: Words are then tagged as being inside, outside, or at the beginning of a syntactic phrase (using IOB tagging). These tags are then joined to form base phrases.
  3. Semantic Role Labeling: Finally, base phrases are classified into semantic roles (arguments) relative to a predicate. This stage uses features derived from POS tags, syntactic chunk labels, and previous semantic tag decisions, all within a context window.

This approach contrasts with "word-by-word" (W-by-W) methods that classify each word individually. The P-by-P method is designed to be computationally less expensive and require simpler annotations.

Datasets

The experiments were conducted using sections 15-18 of the PropBank data. The authors used a tool called chunklink to flatten syntactic trees from PropBank and then derived the new corpus structure introduced in the paper. Sections 00 and 23 of PropBank were used for development and testing.

Key Insights and Ideas

  • Shallow, Non-Overlapping Trees are Sufficient: The paper demonstrates that a shallow, non-overlapping tree structure, enriched with lexicalization, can effectively capture predicate-argument relationships without needing full syntactic parse trees.
  • Phrase-by-Phrase is Better than Word-by-Word: The P-by-P approach outperforms the W-by-W method, suggesting that processing larger linguistic units (phrases) leads to more accurate semantic role assignments.
  • Bottom-Up Tagging is Efficient and Portable: Framing semantic chunking as a sequence of tagging actions, particularly in a bottom-up fashion, results in a lightweight, fast, and potentially more portable system. This is especially beneficial for languages lacking robust full syntactic parsers.
  • Annotation Simplicity: The proposed flat annotation scheme for the shallow tree structure is significantly less demanding than traditional treebank annotation, making it easier to scale to new domains and languages.

Key Results

The paper presents a comparison between their proposed Phrase-by-Phrase (P-by-P) method and a Word-by-Word (W-by-W) approach.

  • Performance: The P-by-P method achieved a higher F1 score (59%) compared to the W-by-W method (53%) on a reduced set of 19 PropBank arguments. Precision and recall were also higher for P-by-P.
  • Speed and Efficiency: The P-by-P approach is noted to run faster and use a wider linguistic context, contributing to its lightweight nature.
  • Component Accuracy: The underlying POS tagger achieved 95.5% accuracy, and the phrase chunker achieved an F-metric of 94.5%.

Limitations and Future Work

  • Performance Gap: The authors acknowledge a significant drop in performance for "chunked" semantic analysis compared to mid-90s syntactic and lexical analyses, indicating that semantic extraction remains a challenging task.
  • Feature Engineering: The paper suggests that feature engineering for semantic chunking is an open-ended area, and improvements can be made by incorporating more features, such as word senses, usages, and world knowledge.
  • Training Data: The relatively small size of training sets is identified as a limitation, with the potential for 10-15% absolute performance improvement with more data and features.
  • Extended Structure: While the paper focuses on a shallow structure, the authors note that exploring deeper structures using simple rules at the phrasal level is a worthy direction for future research.
  • Portability: While the proposed structure supports flat annotation for easier porting, further work is needed to fully realize its potential across different domains and languages.

Summary updated

BibTeX
@inproceedings{hacioglu2004lightweight,
  author = {Kadri Hacioglu},
  title = {{A Lightweight Semantic Chunker Based on Tagging}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2004},
}
Figure