Word Segmentation on Chinese Mirco-Blog Data with a Linear-Time Incremental Model

Kaixu Zhang, Maosong Sun, Changle ZhouCIPS-SIGHAN 2012 · OpenReview

TL;DR

This paper introduces a linear-time incremental model for Chinese micro-blog word segmentation, a challenging task due to the informal and noisy nature of micro-blog text. By incorporating rich features and a novel preprocessing step, the model achieves competitive performance on a specialized micro-blog dataset, highlighting the importance of tailored features for this domain.

What Problem Does This Paper Address?

Chinese word segmentation is a fundamental task in Natural Language Processing (NLP), but it's significantly more complex than in languages like English because Chinese characters are not separated by spaces. This means a single string of characters needs to be parsed into meaningful words. While research has extensively focused on news articles and formal text, the rise of micro-blogging platforms like Weibo has introduced a new challenge. Micro-blog text is characterized by its informality, brevity, use of slang, abbreviations, URLs, and special characters, making traditional segmentation models less effective. This paper aims to bridge this gap by developing a robust word segmentation model specifically for Chinese micro-blog data.

Methods

The core of the paper is a linear-time incremental model for word segmentation. This approach treats segmentation as an action sequence generation problem. For each character in the input sentence, the model decides whether to "separate" (s) it from the next character (indicating the end of a word) or "combine" (c) it (indicating it's part of the same word). This decision-making process is guided by a feature vector and a learned weight vector.

What's novel here is the adaptation of these techniques for micro-blog data:

  • Linear-Time Incremental Parsing: The model builds upon existing work on linear-time incremental parsing, allowing for efficient processing.
  • Rich Feature Engineering: The model employs a wide array of features, including:
  • Character-based features: Information about individual characters and their types (Chinese, Latin, numeric).
  • Word-based features: Leveraging existing lexicons (idioms, general word lists) and frequency information.
  • Tagger-based features: Crucially, they train a general-purpose joint word segmentation and Part-of-Speech (POS) tagging model on a larger corpus (People's Daily) and use its output as features for the micro-blog model. This "annotation adaptation" helps overcome the scarcity of labeled micro-blog data.
  • Preprocessing: A rule-based preprocessing step is introduced to handle common micro-blog elements like URLs, "@" mentions, "#" tags, and punctuation. These are pre-assigned actions (s or c) to reduce the search space for the main model.

Datasets

The paper evaluates its model on two types of datasets:

  • SIGHAN 2005 Bake-off Datasets: The model is tested on four standard datasets (AS, CityU, MSR, PKU) from the 2005 Chinese Word Segmentation Bake-off to compare its performance against existing work on more general Chinese text.
  • Micro-blog Data: A custom dataset of 500 micro-blog sentences is used for training and development. This dataset is noted as being quite small, which is a significant challenge.

Key Insights and Ideas

  • Micro-blog data is fundamentally different: The informal nature and unique characteristics of micro-blogs necessitate specialized approaches, as standard models struggle.
  • Feature engineering is paramount: The success of the model hinges on carefully designed features that capture the nuances of micro-blog text, especially character types and external linguistic resources.
  • Leveraging external knowledge is key for low-resource scenarios: When labeled data is scarce (as with micro-blogs), using models trained on larger, related datasets (like the People's Daily corpus for POS tagging) as feature extractors is a powerful strategy.
  • Preprocessing can significantly improve efficiency: By handling common noise and special elements upfront, the model can focus on the core segmentation task more effectively.

Key Results

  • Competitive performance on SIGHAN 2005: The model achieves an F-score of 0.953 on the AS dataset, comparable to other state-of-the-art models.
  • Strong performance on micro-blog data: The final F-score for the micro-blog data is 0.9478, with 44.88% of sentences segmented correctly.
  • Feature impact: Adding micro-blog specific features (character-type-based, lexical, and tagger-based) significantly improves performance compared to using only basic features. For instance, removing character-type-based features drops the F-score to 0.9383, and removing lexical features drops it further to 0.9201.

Limitations and Future Work

  • Limited micro-blog data: The authors explicitly state that the small size of the annotated micro-blog dataset (500 sentences) is a major bottleneck. They believe performance would improve with more training data.
  • Inconsistent word output: Statistical models can sometimes produce inconsistent segmentations. The authors suggest that a post-processing step could be used to address this.
  • Feature selection: The paper mentions spending considerable time comparing feature combinations, implying that a more sophisticated method for feature selection would be beneficial.
  • Future directions: The authors suggest that more annotated data, semi-supervised methods, and improved post-processing techniques could further enhance performance.

Summary updated

BibTeX
@inproceedings{zhang2012word,
  author = {Kaixu Zhang and Maosong Sun and Changle Zhou},
  title = {{Word Segmentation on Chinese Mirco-Blog Data with a Linear-Time Incremental Model}},
  booktitle = {CIPS-SIGHAN},
  year = {2012},
  url = {https://openreview.net/forum?id=pxPg-hCUzD9},
}
Figure