Abstract
Ziqiang Cao, Sujian Li, Heng Ji. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP).
- This paper introduces DP-4, a four-level Dirichlet Process model that jointly performs Chinese word segmentation (WS), term extraction (TE), and keyword extraction (KE). By breaking the traditional "pipeline" approach, the model allows domain-specific and document-level information to improve segmentation accuracy, effectively mitigating error propagation.
Introduction
In Chinese Natural Language Processing, word segmentation (WS) is the foundational first step because the language lacks explicit word boundaries like spaces. Traditionally, researchers have employed a pipelined framework: first segmenting text into words, then identifying domain-specific "terms" (e.g., thrombocytopenia), and finally extracting document-specific "keywords" (e.g., heparinoid).
However, this sequential approach suffers from two major flaws:
- Error Propagation: Errors in the initial segmentation phase inevitably cascade, causing subsequent term and keyword extraction to fail.
- Information Silos: Pipelined models cannot leverage "downstream" information to improve "upstream" tasks. For instance, knowing a character sequence is a frequent domain-specific term should inform the segmenter to keep it intact, but standard segmenters lack access to this domain-level context.
Methodology: The DP-4 Model
To address these issues, the authors propose a four-level Dirichlet Process (DP-4) model. This Bayesian non-parametric approach captures word distributions across different granularities:
- Levels 1 & 2 (Corpus): Capture general language patterns across the entire dataset.
- Level 3 (Domain): Capture distributions specific to a field (e.g., Physics vs. Medicine).
- Level 4 (Document): Capture distributions specific to a single text.
Technical Innovations
- Joint Learning: Words, terms, and keywords are treated as entities that emerge simultaneously during the sampling process, rather than through sequential steps.
- Sentence-wise Gibbs Sampler: To maintain computational feasibility, the authors developed a sampler that processes entire sentences using the Viterbi algorithm, significantly improving efficiency over character-by-character sampling.
- Scoring Functions: The model identifies terms and keywords by comparing the probability of a word appearing in a specific domain or document against its probability in the general corpus. Words significantly more likely in a specific context are flagged accordingly.
Experimental Setup
The model was evaluated using two primary data sources:
- PKU Corpus: A standard dataset of manually segmented news texts (approx. 1.87 million characters) used to train base corpus-level distributions.
- New 10-Domain Corpus: A newly collected dataset of 2,000 documents across ten domains (e.g., Physics, Medicine, Art, Economy). Each document was manually annotated for gold-standard segmentation, terms, and keywords.
Results and Discussion
The DP-4 model was compared against standard baselines, including Conditional Random Fields (CRFs) and the Hierarchical Dirichlet Process (HDP).
As shown in the results table, DP-4 consistently outperformed existing methods:
- Word Segmentation: DP-4 achieved an F1 score of 0.821, surpassing the state-of-the-art CRF (0.817).
- Term/Keyword Extraction: The improvements were most pronounced here. For Term Extraction (Top 50), DP-4 reached an accuracy of 0.704, compared to the CRF-pipeline's 0.624.
- Out-of-Vocabulary (OOV) Recovery: DP-4 demonstrated a superior ability to handle unseen words (OOV-R of 0.374 vs. CRF's 0.330), proving that joint learning helps the model "discover" new technical terms through contextual feedback.
Conclusion
The DP-4 model demonstrates that joint learning effectively mitigates error propagation in Chinese NLP. By modeling terms and keywords as deviations from a base language distribution, the system creates a virtuous cycle where better term recognition leads to more accurate segmentation. Future work may involve incorporating Part-of-Speech (POS) tags to further refine results and optimizing the model's computational complexity.
Summary updated