An Improved CRF based Chinese Language Processing System for SIGHAN Bakeoff 2007

Xihong Wu, Xiaojun Lin, Xinhao Wang, Chunyao Wu, Yaozhong Zhang, Dianhai YuIJCNLP 2008 · OpenReview

TL;DR

This paper presents an improved Chinese language processing system for the SIGHAN Bakeoff 2007, focusing on word segmentation, named entity recognition, and part-of-speech tagging. The authors enhance standard Conditional Random Fields (CRFs) by integrating n-gram language models to capture higher-level linguistic information and employ a transformation-based learning (TBL) technique for post-processing to correct systematic errors.

What Problem Does This Paper Address?

Chinese language processing presents unique challenges compared to languages with explicit word delimiters like English. Specifically, word segmentation (WS) requires identifying word boundaries in a continuous character stream. Named Entity Recognition (NER) and Part-of-Speech (POS) tagging, while common NLP tasks, are complicated by the inherent ambiguity and lack of explicit word structure in Chinese. Existing models, like standard CRFs, often focus on local dependencies between characters, potentially missing crucial higher-level linguistic context (e.g., word-level language models). Furthermore, even sophisticated models can exhibit systematic errors due to patterns in the data, necessitating robust error correction mechanisms.

Methods

The core of the system relies on Conditional Random Fields (CRFs), a powerful discriminative model for sequence labeling. For the Word Segmentation (WS) and Named Entity Recognition (NER) tasks, the authors enhance CRFs by incorporating an n-gram language model. This integration aims to leverage word-level linguistic information that might be missed by character-level CRFs. The integration is achieved through a log-linear model that combines scores from the CRF and the language model.

To further refine the system's performance, a Transformation-Based Learning (TBL) technique is employed for post-processing. TBL is used to identify and correct systematic errors that often occur in specific patterns, improving accuracy beyond the initial model's capabilities.

For the POS tagging task, CRFs are used for the CTB corpus, while a Maximum Entropy (ME) model is adopted for other corpora due to resource constraints. The ME model is enhanced by incorporating the POS tag of the previous word and utilizing dynamic programming for decoding.

Datasets

The paper describes participation in the SIGHAN Bakeoff 2007, implying the use of standard datasets provided for this competition across the WS, NER, and POS tagging tracks. For the open tracks of WS, additional training data was generated by transforming the People Daily Corpus using auto-extracted transition rules. For the CTB open track in POS tagging, the Chinese Treebank 5.0 corpus was used.

Key Insights and Ideas

  • Bridging the Gap between Character and Word Levels: The integration of n-gram language models with character-based CRFs is a key insight. It highlights the importance of not just local character dependencies but also the statistical properties of word sequences for accurate Chinese NLP.
  • Systematic Error Correction with TBL: The successful application of TBL for post-processing demonstrates its effectiveness in identifying and rectifying recurring error patterns, a common challenge in sequence labeling tasks. This suggests that a multi-stage approach, combining powerful initial models with targeted error correction, can yield significant performance gains.
  • Resource-Aware Model Selection: The choice of ME models for POS tagging on certain corpora due to resource limitations showcases a practical approach to balancing performance and computational feasibility in real-world NLP system development.

Key Results

The paper reports strong performance across various tracks of the SIGHAN Bakeoff 2007. The authors state their systems were ranked 1st in 6 tracks and achieved top-level performance in several others. Specific F-value results are presented in tables for WS (Tables 2 and 3) and NER (Table 4), and accuracy for POS tagging (Table 5). For instance, in closed WS tracks, the +LM+Post approach generally improved upon the baseline and +LM approaches, with F-values reaching up to 95.8% for SXU. In open WS tracks, results also showed improvements, with CTB(a) achieving 99.3% F-value. The POS tagging results show accuracies reaching up to 96.5% for CTB with the +Syntax approach.

Limitations and Future Work

The authors acknowledge that for the open tracks of WS, they did not have enough time to finish parameter estimation on the new data, leading to the use of parameters trained for closed tracks. This sensitivity to parameters when additional training data is introduced is noted as a potential area for improvement. The paper also mentions that the TBL rules were trained via a five-fold cross-validation approach, implying that further tuning of TBL parameters or rule sets could be explored. The paper concludes by emphasizing the effectiveness of their system integration approach and the utility of heuristic strategies for generating additional training data.

Summary updated

BibTeX
@inproceedings{wu2008improved,
  author = {Xihong Wu and Xiaojun Lin and Xinhao Wang and Chunyao Wu and Yaozhong Zhang and Dianhai Yu},
  title = {{An Improved CRF based Chinese Language Processing System for SIGHAN Bakeoff 2007}},
  booktitle = {IJCNLP},
  year = {2008},
  url = {https://openreview.net/forum?id=JBbRETYk5x},
}
Figure