SSMI: Semantic Similarity and Mutual Information Maximization Based Enhancement for Chinese NER

Pengnian Qi, Biao QinAAAI 2023 · DOI

Introduction

Named Entity Recognition (NER)—the process of identifying names, places, and organizations in text—is significantly more challenging in Chinese than in English due to the lack of explicit word boundaries. Traditional lexicon-based models attempt to solve this by matching characters against a dictionary, but they often suffer from "noise"—irrelevant dictionary matches that confuse the model.

This paper introduces SSMI (Semantic Similarity and Mutual Information), a framework that intelligently filters dictionary words based on their semantic relevance. By leveraging Mutual Information Maximization (MIM) to align character, word, and sentence-level features, SSMI effectively reduces noise and achieves state-of-the-art results across major benchmarks.

Methodology

The SSMI framework operates through a three-stage pipeline designed to ensure only the most relevant lexical information is integrated into the model.

Latent Word Boundary Selection

To address the noise inherent in dictionary matching, SSMI performs a preliminary segmentation using BERT. It calculates the cosine similarity between the BERT-derived context and the words found in the dictionary. By keeping only the "best" match and high-confidence candidates, the model filters out semantically irrelevant words before they can propagate errors.

Semantic similarity-based filtering of dictionary words within the embedding space.

Multi-View Enhancement (MIM)

To bridge the granularity gap between characters, words, and sentences, the authors utilize Mutual Information Maximization (MIM). This "Information Augment" module ensures that features at different levels actually complement one another:

  • Local MIM: Calculates the statistical association between specific characters and words.
  • Global MIM: Uses a contrastive learning framework (InfoNCE) to ensure the overall sentence representation is tightly aligned with the selected word boundaries.
The Information Augment framework detailing the Concat Embedding, Local MIM, and Global MIM components.

Neural Architecture

These enhanced features are fed into a sophisticated fusion pipeline consisting of a Convolutional Attention Layer (CAN), a GRU layer for sequence modeling, and a Biaffine layer. The final entity labels are predicted using a Conditional Random Field (CRF) layer.

Experimental Results

The model was evaluated on four diverse Chinese NER benchmarks: Ontonotes 4.0, MSRA, Weibo, and Resume. SSMI achieved State-of-the-Art (SOTA) performance on all four datasets.

  • Social Media Robustness: On the Weibo dataset, SSMI achieved an F1-score of 72.83, a significant jump from the 67.30 BERT-based baseline. This indicates the model is exceptionally good at handling the informal language and noise typical of social media.
  • Large-Scale Performance: On Ontonotes 4.0, it reached an F1 of 83.52, proving its efficacy on formal, large-scale text.

Discussion and Insights

The success of SSMI highlights that context is critical for lexicon filtering. Pre-trained models like BERT are far more effective at "vetting" dictionary matches than simple frequency-based weights. Furthermore, the inclusion of Global MIM was found to be particularly impactful for longer sentences, where complex contextual dependencies require tighter alignment between word-level features and the overall sentence context.

While the multi-stage filtering and MIM alignment add some computational overhead, the use of a parallelizable CNN/Attention-based architecture for fusion makes the model efficient on modern hardware. Future research may explore adapting this MIM-based strategy to other languages where word-level ambiguity persists.

Summary updated

BibTeX
@inproceedings{qi2023ssmi,
  author = {Pengnian Qi and Biao Qin},
  title = {{SSMI: Semantic Similarity and Mutual Information Maximization Based Enhancement for Chinese NER}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2023},
  doi = {10.1609/aaai.v37i11.26580},
}
Figure