Abstract
Chinese Named Entity Recognition (NER) is uniquely challenging due to the absence of explicit word boundaries and the computational bottlenecks associated with traditional lattice-based integration of lexicon information. This paper introduces LADA-Trans-NER, a high-performance framework that combines a novel Lexicon-Attention mechanism with an Adaptive Rank GAN (AR-GAN) for data augmentation. By treating sentences as lexicon-based character graphs and utilizing an adaptive Transformer backbone, the model achieves state-of-the-art results across multiple benchmarks while significantly outperforming previous methods in training efficiency.
Introduction
Unlike English, Chinese does not use spaces to denote word boundaries, making NER a complex task. Traditional approaches often rely on a two-step pipeline: word segmentation followed by entity labeling. However, segmentation errors frequently propagate to the NER stage, degrading performance. While character-based models with lexicon features have become the standard to mitigate this, they often suffer from two major flaws: computational inefficiency due to complex "lattice" structures that hinder parallel processing, and data scarcity, particularly in specialized domains like cybersecurity.
Methodology
The LADA (Lexicon-Attention and Data-Augmentation) framework is built upon three primary pillars designed to enhance both accuracy and speed.
Lexicon-based Character Graphs (LCG)
Rather than a rigid lattice, LADA treats the input sentence as a directed graph where characters serve as nodes and potential dictionary words act as edges. A Lexicon-Attention mechanism identifies the optimal path through this graph, capturing the regularity of word lengths to filter out irrelevant matches. This allows the model to distinguish, for example, a four-character word from four individual characters.
Word-Character Fusion and Transformer Backbone
Characters are enriched with features from four word sets based on the BMES (Begin, Middle, End, Single) tagging scheme. This fusion provides the model with precise context regarding a character's position within potential dictionary words. The core architecture utilizes an Adaptive Transformer encoder modified with relative position encoding to process these fused features efficiently.
Adaptive Rank GAN (AR-GAN)
To address data imbalance and scarcity, the authors implemented an AR-GAN. This component features a Generator that creates synthetic sequences and a Ranker that learns to distinguish real data from synthetic data. This adversarial training strategy forces the model to learn more robust representations, which is particularly beneficial for rare entity types.
Experimental Setup and Datasets
The model was evaluated on four standard Chinese NER benchmarks—Resume, MSRA, Weibo, and OntoNotes v4—alongside a new domain-specific dataset, CTI (Cyber Threat Intelligence). The CTI dataset contains 17 entity types related to cybersecurity, such as malware names and attack types, which often feature complex, nested structures.
Results and Discussion
LADA-Trans-NER achieved state-of-the-art F1 scores across all tested datasets, notably reaching 98.70% on Resume and 95.23% on the CTI dataset.
Efficiency and Robustness
By moving away from pointer-based lattice LSTMs to a graph-based attention approach, the model leverages GPU parallelization, resulting in training speeds approximately 4x faster than traditional methods (processing ~117 sentences/second vs. ~28 sentences/second). Furthermore, the model demonstrated superior performance in handling "entity nesting," where shorter entities are contained within longer ones.
Conclusion
LADA-Trans-NER effectively addresses the efficiency and data scarcity issues prevalent in Chinese NER. By integrating lexicon information through an attention-based graph approach and utilizing GAN-based augmentation, the framework provides a robust and scalable solution for both general and specialized domains. Future work may involve integrating these lexicon-attention mechanisms into larger pre-trained models like BERT or RoBERTa to further enhance performance.
Summary updated