Incremental Joint POS Tagging and Dependency Parsing in Chinese

Jun Hatori, Takuya Matsuzaki, Yusuke Miyao, Jun'ichi TsujiiIJCNLP 2011 · OpenReview

Abstract

This paper introduces the first incremental framework for jointly performing Part-of-Speech (POS) tagging and dependency parsing in Chinese. By integrating these tasks into a single shift-reduce process and employing a novel "delayed features" mechanism to handle look-ahead information, the authors significantly reduce error propagation. The resulting model achieves new state-of-the-art performance on the Chinese Penn Treebank, demonstrating that structural context is essential for resolving linguistic ambiguities.

Introduction

In traditional Natural Language Processing (NLP), tasks are typically handled in a sequential pipeline: POS tagging followed by dependency parsing. This approach has two major flaws, particularly for languages like Chinese. First, it suffers from error propagation, where tagging mistakes lead to parsing failures. Second, it lacks context; many Chinese words are ambiguous (e.g., "的" can be a genitive marker or a complementizer), and distinguishing between them often requires structural information that the tagger does not yet possess.

Example of a Chinese sentence with joint POS tags and dependency arcs.

To address these issues, the authors propose a joint model that processes text left-to-right, allowing tagging and parsing decisions to inform one another simultaneously.

Methodology

The framework builds upon a Shift-Reduce parsing architecture. To enable joint processing, the standard SHIFT action is replaced with SHIFT(t), which assigns a POS tag t to a word as it is moved onto the stack.

Key Innovations

  • Dynamic Programming (DP): To manage the expanded search space of joint actions, DP is used to pack equivalent parser states, maintaining efficiency during beam search.
  • Delayed Features: Standard parsers rely on look-ahead features (tags of upcoming words). In an incremental joint model, these tags are unavailable. The authors developed "delayed feature vectors" that store these features and evaluate their scores only once the relevant words are eventually shifted and tagged.
  • Syntactic Features for Tagging: The model incorporates features that allow the current tagging decision to be influenced by the partial dependency trees already constructed on the stack.
Feature templates used in the joint model, including baseline, distance/valency, look-ahead, and syntactic features.

Experimental Results

The model was evaluated on the Chinese Penn Treebank (CTB) 5. The joint approach (Joint-ZN-) outperformed pipeline baselines across all metrics:

  • Tagging Accuracy: Reached 93.94%, compared to the 93.82% pipeline baseline.
  • Parsing Accuracy (UAS): Achieved 81.33%, a significant improvement over the 78.04% achieved by the pipeline version.

These results confirm that joint modeling helps resolve difficult ambiguities, such as DEC vs. DEG and NN (noun) vs. VV (verb), by providing necessary structural context.

Conclusion

This research presents a computationally viable and highly effective framework for joint POS tagging and dependency parsing in Chinese. By leveraging syntactic context during the tagging phase, the model significantly reduces errors inherent in pipeline systems. Future work may involve integrating word segmentation into this joint process for a more comprehensive end-to-end Chinese NLP system.

Summary updated

BibTeX
@inproceedings{hatori2011incremental,
  author = {Jun Hatori and Takuya Matsuzaki and Yusuke Miyao and Jun'ichi Tsujii},
  title = {{Incremental Joint POS Tagging and Dependency Parsing in Chinese}},
  booktitle = {IJCNLP},
  year = {2011},
  url = {https://openreview.net/forum?id=XM6cbQBVhbT},
}
Figure