Kcr-FLAT: A Chinese-Named Entity Recognition Model with Enhanced Semantic Information

Zhenrong Deng, Yong Tao, Rushi Lan, Rui Yang, Xueyong WangSensors 2023 · DOI

TL;DR

This paper introduces Kcr-FLAT, a novel Chinese Named Entity Recognition (NER) model that significantly improves performance by integrating syntactic information and capturing internal entity regularity. By addressing the inherent ambiguity in Chinese word segmentation and leveraging richer semantic cues, Kcr-FLAT achieves state-of-the-art results on several benchmark datasets.

What Problem Does This Paper Address?

Named Entity Recognition (NER) is a crucial task in Natural Language Processing (NLP) for extracting valuable information from text. While NER for languages like English has seen substantial progress, Chinese NER presents unique challenges. Unlike English, Chinese lacks explicit word delimiters, making word segmentation ambiguous. The same sequence of characters can be interpreted as different words depending on the context, leading to segmentation errors that propagate to downstream tasks like NER. Furthermore, while external lexical resources have been used to enhance Chinese NER, the internal syntactic structure of sentences and the inherent naming regularities within entities themselves have been relatively underexplored. This paper aims to bridge this gap by developing a model that effectively leverages both syntactic information and internal entity regularity to overcome these challenges.

Methods

The Kcr-FLAT model builds upon the Flat-Lattice Transformer (FLAT) architecture and introduces three key enhancements:

  • Key-Value Memory Network (KVMN) for Syntactic Embedding: The model first extracts three types of syntactic information: Part-of-Speech (POS) labels, syntactic constituents, and dependency relations. This syntactic data, along with contextual features, is then encoded using a KVMN. This module allows for the weighted encoding of syntactic information based on its relevance to the context, effectively capturing richer semantic nuances.
  • Cross-Transformer for Semantic Fusion: To address segmentation errors introduced by lexical information and to fuse it with the enhanced syntactic information, a cross-transformer module is employed. This module allows for interaction between lexical embeddings (from lattice embedding) and syntactic embeddings, enabling a more robust integration of these two crucial information sources.
  • Regularity Perception Module: This novel module is designed to capture the internal naming regularities of entities. Chinese entities often follow specific patterns (e.g., "XX River" for locations). This module analyzes spans of text to identify these regularities, further refining entity recognition by considering inherent structural patterns within the entities themselves. This module replaces the traditional linear and CRF layers at the top of the cross-transformer.

Datasets

The proposed Kcr-FLAT model was evaluated on three widely-used Chinese NER datasets:

  • MSRA: Primarily contains news datasets and is a standard benchmark for Chinese NER.
  • Resume: Contains a large number of names of people, places, and companies, often with complex entity structures.
  • Weibo: A dataset derived from micro-blogging platforms, presenting a more diverse and contextually rich set of named entities across various scenarios.

Key Insights and Ideas

  • Syntactic Information is Crucial for Ambiguity Resolution: The paper demonstrates that explicitly encoding syntactic information (POS, constituents, dependencies) is highly effective in resolving the ambiguity inherent in Chinese word segmentation, leading to improved entity boundary detection.
  • Internal Entity Regularity Matters: Capturing the inherent naming patterns within entities themselves provides a powerful signal for accurate classification, especially in cases where lexical and syntactic cues might be less discriminative.
  • Fusion is Key: The success of Kcr-FLAT lies in the effective fusion of lexical, syntactic, and regularity information. The cross-transformer and regularity perception modules play vital roles in integrating these diverse signals.
  • Contextually Rich Datasets Benefit Most: The model shows significant improvements on the Weibo dataset, highlighting the advantage of Kcr-FLAT in handling complex, context-dependent language found in social media.

Key Results

The Kcr-FLAT model achieved state-of-the-art performance on the evaluated datasets. Specifically:

  • On the MSRA dataset, Kcr-FLAT achieved an F1 score of 96.51%.
  • On the Resume dataset, it reached an F1 score of 96.81%.
  • On the Weibo dataset, it obtained an F1 score of 70.12%.

Ablation studies showed that each proposed improvement strategy (KVMN, Cross-Transformer, Regularity Perception) contributed to performance gains. The KVMN for syntactic information provided the most significant improvement, particularly on the Weibo dataset. The final Kcr-FLAT model outperformed the baseline FLAT model by approximately 3% on MSRA, 2% on Resume, and 6.5% on Weibo.

Limitations and Future Work

The authors acknowledge that while their model performs well, it still falls slightly short of the performance of Zhu et al. on the Weibo dataset, who utilized a boundary-smoothing method. They suggest that for variable-scene texts, regularity might limit recognition accuracy in certain contexts.

Future research directions include:

  • Developing more efficient methods for extracting syntactic information without relying on external NLP tools.
  • Exploring better ways to integrate external lexical information and internal regularity to further enhance Chinese NER performance.
  • Extending the proposed model to other NLP tasks.

Summary updated

BibTeX
@article{deng2023kcr,
  author = {Zhenrong Deng and Yong Tao and Rushi Lan and Rui Yang and Xueyong Wang},
  title = {{Kcr-FLAT: A Chinese-Named Entity Recognition Model with Enhanced Semantic Information}},
  journal = {Sensors},
  year = {2023},
  doi = {10.3390/s23041771},
}
Figure