Introduction to CKIP Chinese Word Segmentation System for the First International Chinese Word Segmentation Bakeoff

Wei-Yun Ma, Keh-Jiann ChenSIGHAN 2003 · OpenReview

TL;DR

This paper introduces the CKIP Chinese word segmentation system, which achieved strong performance in the first International Chinese Word Segmentation Bakeoff, particularly on the HK tracks. The system excels at resolving segmentation ambiguities and identifying unknown words, crucial challenges in Chinese NLP.

What Problem Does This Paper Address?

Chinese word segmentation is a fundamental task in Natural Language Processing (NLP), but it's significantly more challenging than in languages like English. Unlike English, Chinese doesn't use spaces to delineate words. This means a single sequence of characters can be segmented in multiple ways, leading to ambiguity. For example, "完成鑑定報告" could be segmented as "完成 鑑定 報告" (complete authenticate report) or "完成鑑定 報告" (complete authentication report). Furthermore, identifying "unknown words" – words not present in the system's lexicon – is another major hurdle. This paper addresses these two core problems: disambiguating word boundaries and effectively detecting and extracting unknown words.

Methods

The CKIP system employs a two-stage approach:

  1. Word Matching and Disambiguation:
    • Word Matching: The system uses a longest matching algorithm to identify potential word segments from left to right.
  2. Disambiguation: When ambiguities arise (multiple possible segmentations for a sequence), a set of heuristic rules is applied. These include:
  3. Longest Matching Rule: Prioritizes the segmentation with the longest word sequence.
  4. Word Length Rule: Selects the segmentation where the standard deviation of word lengths is smallest.
  5. Morphemic Rules: Favors segmentations with fewer bound morphemes or fewer characters in compound words.
  6. Probability Rule: Chooses segmentations with high-frequency monosyllabic words or the highest probability value.
  7. Unknown Word Detection and Extraction:
    • POS Tagging: After initial segmentation, a POS bi-gram tagging model is applied.
  8. Unknown Word Detection: Syntactic discriminators, trained on a segmented corpus, are used to identify monosyllabic morphemes that might be part of an unknown word.
  9. Unknown Word Extraction: A bottom-up merging algorithm, combining statistical and linguistic information, is used to extract unknown words by combining these detected morphemes with their neighbors.

The system also includes adaptations for different segmentation standards across corpora (e.g., handling GB vs. BIG5 character encoding for the PK track).

Datasets

  • Training Data: Specific training corpora for the PK and HK tracks were used. The CKIP lexicon was also utilized, particularly for the open test tracks.
  • Evaluation Data: The system was evaluated on the Beijing University (PK) and Hong Kong Cityu (HK) corpora for both open and closed test tracks as part of the International Chinese Word Segmentation Bakeoff. An additional evaluation was performed on the Academia Sinica (AS) closed track.

Key Insights and Ideas

  • The Dual Challenge: The paper reinforces that both ambiguity resolution and unknown word identification are critical and intertwined challenges in Chinese word segmentation.
  • Heuristic Rules are Powerful: The longest matching rule alone resolves a significant majority (93.21%) of ambiguities, highlighting the effectiveness of simple, yet well-chosen, heuristics.
  • Unknown Word Extraction is Key to Performance: The authors suggest their superior performance, especially on the HK tracks, is largely due to their effective unknown word extraction methods. This is crucial because errors in unknown word handling can cascade into multiple segmentation errors.
  • Corpus Standards Matter: Differences in segmentation standards between corpora (like HK and PK) and character encoding (GB vs. BIG5) significantly impact system performance, necessitating adaptation strategies.

Key Results

  • HK Performance: The CKIP system achieved top ranks in F-scores for both the HK closed and open tracks, with F-scores of 0.940 and 0.956, respectively.
  • PK Performance: The system performed acceptably on the PK tracks, ranking 6th out of 10 systems in the closed track (F-score 0.936) and 3rd out of 8 in the open track (F-score 0.938).
  • AS Performance: On the AS closed track, the system achieved a high F-score of 0.967, suggesting it would have ranked highly if it had participated.
  • Unknown Word Handling: The system demonstrated strong recall on unknown words (Roov), particularly in the HK open track (0.788), contributing to its overall high F-scores.

Limitations and Future Work

  • Character Encoding Issues: The paper identifies character coding conversion (GB to BIG5) as a significant source of errors for the PK track. The authors suggest that re-implementing a GB segmentation version without conversion would be the most direct solution.
  • Segmentation Standards: The variation in segmentation standards across different corpora, some of which are not publicly available, poses a challenge. The authors advocate for a uniform word segmentation standard in the future.
  • Error Types: The authors suggest that "crossing errors" (where segmentation boundaries overlap incorrectly) are more severe than non-crossing errors and should be considered in evaluation metrics.
  • Future Work: While not explicitly detailed, the success in unknown word extraction suggests further refinement in this area could yield even better results. Adapting the system to handle diverse segmentation standards more robustly is also implied.

Summary updated

BibTeX
@inproceedings{ma2003introduction,
  author = {Wei-Yun Ma and Keh-Jiann Chen},
  title = {{Introduction to CKIP Chinese Word Segmentation System for the First International Chinese Word Segmentation Bakeoff}},
  booktitle = {SIGHAN},
  year = {2003},
  url = {https://openreview.net/forum?id=-CP6mY2dPhu},
}
Figure