AHVE-CNER: Aligned Hanzi Visual Encoding Enhance Chinese Named Entity Recognition with Multi-Information

Xuhui Zheng, Zhiyuan Min, Bin Shi, Hao WangCOLING 2025 · OpenReview

Introduction

Named Entity Recognition (NER) in Chinese presents unique challenges compared to alphabetic languages. The lack of explicit word boundaries and the ideographic nature of Chinese characters (Hanzi) mean that semantic clues are often embedded within the visual structure of the characters themselves. While previous "glyph-based" models have attempted to use either image-based CNN features or radical-based decompositions, they often fail to connect pictorial elements with abstract radicals.

Crucially, characters sharing identical radicals can have vastly different meanings based on their spatial arrangement. For example, the characters "叶" (leaf) and "古" (ancient) share the same components but arrange them differently. AHVE-CNER addresses this by explicitly aligning visual patches of a character image with its corresponding radical components.

Figure 1: Illustration of character composition and spatial arrangement. (a) shows characters 'shao' and 'kao' sharing the 'fire' radical. (b) demonstrates how 'gu' and 'ye' use identical components in different spatial configurations, a distinction AHVE-CNER is designed to capture.

Methodology: The AHVE-CNER Framework

AHVE-CNER introduces a multi-modal framework that explicitly aligns character images with their internal radical structures across three main stages:

1. Multi-Source Encoding

  • Contextual Semantic: A pre-trained BERT model captures sentence-level context.
  • Phonetic (Pinyin): A CNN processes character initials, finals, and tones to create pronunciation embeddings, providing a strong signal for disambiguation.
  • Visual Alignment: Using a Vision Transformer (ViT), the model divides character images into patches and aligns them with radical embeddings from a dictionary. This grounded structural representation allows the model to distinguish between characters with similar components but different meanings.
Figure 2: Visual radical identification in the characters 'shao' and 'kao'. The model identifies the 'fire' radical (highlighted in red) as a key semantic component.

2. Fusion via Gated Pangu-π Cross Transformer

Instead of simple concatenation, AHVE-CNER employs a fusion block inspired by the Pangu-π architecture. It utilizes Augmented Shortcuts and Series Activation Functions (SIAF) to capture complex inter-modal relationships. A Tri-modal Gating unit dynamically weights text, visual, and phonetic information for each character based on context.

3. Decoding

The framework uses W2NER to treat NER as a word-word relationship classification task, making it effective for both flat and nested entities.

Experimental Results

The model was evaluated on the Weibo (informal social media) and Resume (formal) benchmarks, setting new performance standards:

  • Weibo: Achieved an F1-score of 73.09%, a 2.3% improvement over the previous state-of-the-art VisPhone.
  • Resume: Reached an F1-score of 97.02%, outperforming other glyph-based models like FGN and MECT.

Ablation studies highlighted that while visual alignment is crucial for handling rare characters and spatial disambiguation, phonetic information remains a vital signal in modern Chinese NER.

Conclusion

AHVE-CNER demonstrates that aligning visual patches with radical components significantly enhances a model's ability to understand Hanzi structure. While the added complexity increases latency by approximately 27% compared to text-only models, the substantial gains in accuracy across diverse datasets establish a new state-of-the-art for Chinese NER. Future work will explore integrating word-level lexicons to further refine the model's performance.

Summary updated

Abstract

The integration of multi-modal information, especially the graphic features of Hanzi, is crucial for improving the performance of Chinese Named Entity Recognition (NER) tasks. However, existing glyph-based models frequently neglect the relationship between pictorial elements and radicals. This paper presents AHVE-CNER, a model that integrates multi-source visual and phonetic information of Hanzi, while explicitly aligning pictographic features with their corresponding radicals. We propose the Gated Pangu-𝜋 Cross Transformer to effectively facilitate the integration of these multi-modal representations. By leveraging a multi-source glyph alignment strategy, AHVE-CNER demonstrates an improved capability to capture the visual and semantic nuances of Hanzi for NER tasks. Extensive experiments on benchmark datasets validate that AHVE-CNER achieves superior performance compared to existing multi-modal Chinese NER methods. Additional ablation studies further confirm the effectiveness of our visual alignment module and the fusion approach.
BibTeX
@inproceedings{zheng2025ahve,
  author = {Xuhui Zheng and Zhiyuan Min and Bin Shi and Hao Wang},
  title = {{AHVE-CNER: Aligned Hanzi Visual Encoding Enhance Chinese Named Entity Recognition with Multi-Information}},
  booktitle = {International Conference on Computational Linguistics},
  year = {2025},
  url = {https://openreview.net/forum?id=zHck4NyvVK},
}
Figure