Unsupervised Training for Overlapping Ambiguity Resolution in Chinese Word Segmentation

Mu Li, Jianfeng Gao, Changning Huang, Jianfeng LiSIGHAN 2003 · DOI

TL;DR

This paper introduces an unsupervised method for resolving overlapping ambiguities in Chinese word segmentation, a crucial but challenging task. By training an ensemble of Naïve Bayesian classifiers on an unlabelled corpus, the authors achieve state-of-the-art accuracy, rivaling supervised methods without requiring manual annotation.

What Problem Does This Paper Address?

Chinese word segmentation is the process of identifying word boundaries in a continuous stream of characters, unlike languages with explicit spaces. A significant challenge in this task is "overlapping ambiguity" (OA), where a sequence of characters can be segmented in multiple valid ways, with the ambiguous words overlapping. For instance, "各国" (ge4-guo2, every country) and "国有" (guo2-you3, state-owned) can both be part of a segmentation of "各国.有" (ge4-guo2-you3, every country has). Resolving these ambiguities is vital for downstream Natural Language Processing (NLP) tasks. Traditional methods often rely on large, manually annotated datasets, which are expensive and time-consuming to create. This paper aims to overcome this limitation by developing an unsupervised approach to tackle overlapping ambiguities.

Methods

The core of the proposed method is an ensemble of adapted Naïve Bayesian classifiers. These classifiers are trained in an unsupervised manner using an unlabelled Chinese text corpus. The key innovation lies in the training procedure:

  • Ensemble of Naïve Bayesians: Instead of relying on a single classifier, the paper uses an ensemble of classifiers, each trained with different context window sizes (left and right). This allows for capturing a broader range of contextual information.
  • Unsupervised Training: The main challenge in unsupervised training is dealing with potential errors in the training data, especially overlapping ambiguities themselves. The authors address this by identifying all "Overlapping Ambiguous Strings" (OASs) in the training data and replacing them with a special token [GAP]. This effectively removes potentially erroneous data points from the training set.
  • Feature Engineering: The classifiers use context words within varying window sizes as features. The probability of a segmentation is then calculated based on the probability of the segmentation itself and the conditional probabilities of the context words given the segmentation.
  • Majority Vote: The final decision for an ambiguous string is made by a majority vote among the ensemble of classifiers.

Datasets

  • Training Data: The paper utilizes a large, unlabelled Chinese text corpus from "People's Daily" news articles (year 2000), containing over 24 million characters. This corpus is processed to remove OASs before training.
  • Test Data: A manually annotated test set of 5,759 longest OASs from "People's Daily" news articles (year 1997) was used for evaluation. This dataset is distinct from the training data.
  • Lexicon: A lexicon of 93,700 entries was used.

Key Insights and Ideas

  • Unsupervised Learning for Ambiguity Resolution: The paper demonstrates that it's possible to achieve high accuracy in resolving complex linguistic ambiguities like overlapping segmentation without relying on manually labelled data. This significantly reduces the cost and effort required for building such systems.
  • The Power of Ensemble Methods: Combining multiple classifiers, even simple ones like Naïve Bayesians, with different feature sets (context window sizes) leads to a significant improvement in performance compared to individual classifiers.
  • Strategic Data Cleaning: The method of replacing OASs with a special token [GAP] in the training data is a clever way to mitigate the negative impact of ambiguous examples on classifier training. This "self-cleaning" mechanism is crucial for unsupervised learning in this domain.
  • Context Matters (but not always): While context-sensitive OASs benefit greatly from wider context windows, the paper also notes that some OASs can be resolved without context information, suggesting a potential for hybrid approaches.

Key Results

The proposed unsupervised approach achieved an accuracy of 94.3% on the manually annotated test set. This performance is comparable to, and in some cases rivals, rule-based and supervised training methods. The ensemble consistently outperformed individual classifiers, with the best individual classifier achieving around 89% accuracy on a subset of the data.

Limitations and Future Work

  • Oracle Accuracy: The paper acknowledges that not all OASs can be perfectly resolved even with perfect knowledge (oracle accuracy), as some ambiguities are inherently difficult to disambiguate. The proposed method's accuracy is close to this theoretical upper bound.
  • Further Improvements: The authors suggest that more powerful ensemble strategies could further improve performance. They also propose investigating ways to combine their unsupervised approach with Sun's (1997) method, which uses context-free OASs, to potentially enhance accuracy.
  • Limited Context-Free OASs: While the method is largely unsupervised, the authors mention that using a core set of manually labeled context-free OASs could be a promising direction for further improvement.

Summary updated

BibTeX
@inproceedings{li2003unsupervised,
  author = {Mu Li and Jianfeng Gao and Changning Huang and Jianfeng Li},
  title = {{Unsupervised Training for Overlapping Ambiguity Resolution in Chinese Word Segmentation}},
  booktitle = {SIGHAN},
  year = {2003},
  doi = {10.3115/1119250.1119251},
}
Figure