Neural Character-level Dependency Parsing for Chinese

Haonan Li, Zhisong Zhang, Yuqi Ju, Hai Zhao

Traditional Chinese Natural Language Processing (NLP) relies on word segmentation as a prerequisite, leading to error propagation and the loss of internal word structure. This paper, presented at AAAI-18, introduces a neural character-level dependency parser that bypasses word segmentation. By utilizing a Stack LSTM architecture and linguistically motivated character-level POS tags, the model achieves state-of-the-art performance and provides a deeper structural understanding of Chinese sentences.

The Wordhood Problem in Chinese NLP

In languages like English, spaces provide clear word boundaries. Chinese, however, is written as a continuous stream of characters. Traditionally, Chinese NLP requires a pre-processing step called Chinese Word Segmentation (CWS). This creates two major issues:

  1. Error Propagation: If the segmenter makes a mistake (e.g., splitting "Western medicine" into three separate characters instead of one or two units), every subsequent step—like part-of-speech (POS) tagging and dependency parsing—is likely to fail.
  2. Vague Definitions: There is no universal standard for what constitutes a "word" in Chinese. Furthermore, word-level parsing ignores the internal "syntax" of a word—how characters within a word relate to each other—which often contains vital semantic information.
Comparison of word-level, word-internal, and character-level dependency trees.

As shown in the figure above, character-level dependency trees (c) unify word-level relationships (a) and word-internal structures (b) into a single, cohesive representation.

Methodology: Neural Character-Level Parsing

The authors propose a Neural Character-Level Dependency Parser that treats characters as the primary units of syntax, bypassing the need for error-prone word segmentation.

Comparison of the proposed neural model against traditional and non-neural joint models.

Model Architecture

  • Stack LSTM Parser: The model utilizes a transition-based parser powered by a Stack LSTM. This architecture is particularly well-suited for character-level tasks because LSTMs can maintain long-range dependencies across the longer sequences generated when a sentence is broken into characters rather than words.
  • Unified Framework: Instead of a multi-stage pipeline, the model streamlines the process into character POS tagging followed by character-level parsing.
  • Linguistic POS Tagging: A key innovation is the use of detailed character-level POS tags. The authors found that a hybrid approach (ROOT-CHAR+WORD) performed best. This strategy assigns word-level tags to all characters but adds specific character-level tags to the "root" character of a word.
  • Character Embeddings: The model uses character embeddings as the base representation, allowing it to learn semantic similarities between characters directly.

Datasets and Evaluation

The study relies on two primary data sources: the Chinese Penn Treebank 5.1 (CTB5) for word-level dependencies and the SJTU Chinese Character Dependency Treebank (SCDT), which provides human-annotated internal dependency structures for the words in CTB. By combining these, the authors created a "full" character-level treebank.

Key Results

  • Accuracy: The best tagging strategy (ROOT-CHAR+WORD) achieved a Unlabeled Attachment Score (UAS) of 89.51% and a Labeled Attachment Score (LAS) of 87.45% on character-level tasks.
  • Neural vs. Traditional: The neural LSTM parser significantly outperformed traditional non-neural transition-based parsers (like MaltParser) across all metrics.
  • Word-Level Restoration: To compare with traditional models, the authors restored word-level trees from their character trees, achieving a 79.44% UAS, proving competitive with previous joint-learning models.
  • Robustness: In tests using predicted rather than "gold" inputs, the character-level parser showed much better resistance to error propagation compared to word-level parsers.

Discussion and Future Work

The authors demonstrate that words have an "internal syntax" that is vital for deep understanding. By parsing at the character level, the model becomes immune to standard segmentation pitfalls. However, performance still leans heavily on the quality of the character-level POS tagger. Future work may involve improving character POS tagging and developing more integrated, learnable approaches for "word restoration" to further improve word-level metrics.

Summary updated

Abstract

This paper presents a truly full character-level neural dependency parser together with a newly released character-level dependency treebank for Chinese, which has suffered a lot from the dilemma of defining word or not to model character interactions. Integrating full character-level dependencies with character embedding and human annotated character-level part-of-speech and dependency labels for the first time, we show an extra performance enhancement from the evaluation on Chinese Penn Treebank and SJTU (Shanghai Jiao Tong University) Chinese Character Dependency Treebank and the potential of better understanding deeper structure of Chinese sentences.
BibTeX
@inproceedings{li2018neural,
  author = {Haonan Li and Zhisong Zhang and Yuqi Ju and Hai Zhao},
  title = {{Neural Character-level Dependency Parsing for Chinese}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2018},
  doi = {10.1609/aaai.v32i1.12002},
}
Figure