Abstract
This paper presents a hybrid statistical model for Chinese Named Entity Recognition (NER) that integrates fine-grained word features with coarse-grained Part-of-Speech (POS) features. By combining these statistical methods with heuristic human knowledge—including surname lists, organization templates, and positional sub-classes—the proposed system achieves high accuracy in identifying person, location, and organization names. Experimental results on the People’s Daily corpus demonstrate that the hybrid approach significantly outperforms single-feature models, particularly in handling the unique linguistic challenges of the Chinese language.
Introduction
Named Entity Recognition (NER) is a foundational task in Natural Language Processing, yet it presents unique challenges in the context of the Chinese language. Unlike English, Chinese lacks orthographic cues such as capitalization to signal proper nouns. Furthermore, the absence of word boundaries necessitates a pre-processing step of word segmentation; errors in this stage frequently propagate to the NER system, leading to recognition failures.
Previous research has typically utilized either "coarse" features (POS tags) or "fine" features (specific words). While POS tags provide structural robustness, they often lack the specificity needed for precise identification. Conversely, word-based models are precise but suffer from data sparseness when encountering unseen entities. This study proposes a hybrid model to bridge this gap, leveraging the strengths of both feature sets to create a more resilient recognition framework.
Methodology
The core of the proposed system is a Hybrid Model that balances word-level and POS-level statistical information. The model calculates the optimal entity sequence by maximizing a joint probability weighted by a balancing factor ($\zeta$).
Positional Sub-classes
A significant innovation in this approach is the use of 19 positional sub-tags instead of generic entity labels. For instance, rather than labeling a sequence simply as a "Person," the model identifies specific components such as the "First character of a Chinese surname" or the "Middle character of a Transliterated Name." This granularity allows the model to capture the internal structural patterns of different entity types, effectively modeling the "skeleton" of Chinese names.
Heuristic Integration and Sub-models
To further improve performance, the authors integrated heuristic human knowledge into the statistical framework to prune the search space and eliminate noisy candidates:
- Specialized Sub-models: Recognizing that foreign names have distinct character distributions, specific sub-models were developed for Japanese and Euramerican transliterations.
- Knowledge Lists: The system utilizes extensive lists of Chinese and Japanese surnames (476 and 9,189 respectively) and transliteration character sets.
- Trigger Keywords and Templates: Heuristic rules identify "trigger" words for locations (e.g., "at", "go to") and organizations (e.g., "company", "inc"). Templates are used to recognize nested organizational structures, such as
[Location] + [Word] + [OrgKeyword].
Experimental Results
The model was evaluated using the People’s Daily (Renmin Ribao) corpus and the MET-2 evaluation set. The hybrid model, optimized with a balancing factor of $\zeta = 2.8$, consistently outperformed models relying on only one feature type.
As shown in Table 2, the Hybrid Model achieved an F-Measure of 94.63% for Person Names (PN), 93.73% for Location Names (LN), and 85.76% for Organization Names (ON). The integration of heuristic knowledge was particularly impactful; the authors noted that adding these "linguistic guardrails" improved the F-Measure for Person Names by 14.8% and for Organizations by 13.8% compared to a purely statistical baseline. This synergy suggests that POS tags act as a safety net, allowing the system to recognize the structure of an entity even when the specific words are unknown.
Conclusion
The fusion of fine-grained word features, coarse-grained POS features, and heuristic linguistic knowledge provides a robust framework for Chinese NER. The hybrid model effectively mitigates the issues of data sparseness and segmentation errors. Future work will focus on the recognition of abbreviated organization names, which remain a challenge due to their irregular and highly compressed forms in technical and news text.
Summary updated