Exploiting Multiple Embeddings for Chinese Named Entity Recognition

Canwen Xu, Feiyang Wang, Jialong Han, Chenliang Li

Introduction

Named Entity Recognition (NER) in Chinese presents unique challenges compared to English. The language lacks natural word delimiters like spaces and conventional linguistic cues such as capitalization. Furthermore, the semantic meaning of Chinese text is distributed across multiple levels: words, characters, and even sub-character components known as "radicals." These radicals often provide strong semantic hints; for instance, characters containing the "disease" radical are rarely part of personal names. Existing models frequently struggle with the noisy, colloquial nature of social media text and fail to fully leverage this rich, multi-level hierarchy.

Methods

The proposed ME-CNER (Multiple Embeddings for Chinese NER) framework addresses these challenges by fusing information from three distinct granularities into a comprehensive character representation:

  1. Radical Embedding: A Convolutional Neural Network (CNN) extracts local context features from character radicals, which act as semantic anchors for rare or unseen characters.
  2. Character Embedding (Conv-GRU): A novel Conv-GRU network processes characters by passing them through a Gated Recurrent Unit (GRU) to capture long-range dependencies, followed by a CNN to extract local features.
  3. Word Embedding: Text is segmented into words using standard toolkits, and these embeddings are aligned with their constituent characters to provide broader context.
Architecture of the ME-CNER model integrating character, radical, and word-level information.

These three representations are concatenated and processed by a BiGRU-CRF (Bidirectional GRU with a Conditional Random Field) tagger for sequence labeling. To further boost performance, character and word embeddings are initialized with pre-trained vectors from large-scale external corpora.

Datasets

The model was evaluated on two distinct benchmarks to test its versatility:

  • Weibo NER Dataset: A small, noisy collection of microblog text containing both named entities and nominal mentions, representative of social media.
  • MSRA News Dataset: A large, formal dataset consisting of clean, structured news text.

Key Results and Insights

ME-CNER demonstrates significant improvements, particularly in challenging, low-resource domains:

  • Social Media Superiority: On the noisy Weibo dataset, ME-CNER achieved an F1 score of 68.93, significantly outperforming the previous state-of-the-art Lattice LSTM (58.79). Even when the baseline was bolstered with identical pre-trained embeddings, ME-CNER maintained a substantial lead of 4.84 F1 points.
  • Efficiency and Scalability: On the formal MSRA News dataset, the model achieved a competitive F1 of 91.45. While slightly behind the Lattice LSTM in raw accuracy on formal text, ME-CNER is over twice as fast to train (<0.5x training time) because its architecture allows for standard batch parallelism.
Performance comparison of ME-CNER against baseline models and ablation study results.

Ablation studies (shown in the table above) confirm that the Conv-GRU and radical embeddings are critical components. Removing radicals or replacing the Conv-GRU with standard CNN or BiLSTM layers leads to a measurable drop in performance, proving that the hybrid architecture effectively marries sequential awareness with local feature extraction.

Conclusion

ME-CNER highlights the power of multi-granularity embeddings for Chinese NER. By treating radicals as semantic anchors and utilizing a hybrid Conv-GRU architecture, the model achieves high performance on noisy data while remaining computationally efficient. While the advantages of radical features are most pronounced in noisy settings, the overall framework provides a robust and scalable solution for modern Chinese NLP tasks.

Summary updated

BibTeX
@inproceedings{xu2019exploiting,
  author = {Canwen Xu and Feiyang Wang and Jialong Han and Chenliang Li},
  title = {{Exploiting Multiple Embeddings for Chinese Named Entity Recognition}},
  booktitle = {ACM International Conference on Information and Knowledge Management},
  year = {2019},
  doi = {10.1145/3357384.3358117},
}
Figure