Introduction
Named Entity Recognition (NER) is a fundamental task in Natural Language Processing, typically framed as a sequence labeling problem using tagging schemes like BIO (Begin, Inside, Outside). In Chinese NER, this task is particularly challenging due to the lack of explicit word delimiters. Standard approaches often suffer from what the authors term Entity Representation Segmentation.
As shown in Figure 1, standard fine-tuning forces the model to treat B- and I- tags as distinct, isolated classes. This leads to fragmented vector representations for tokens belonging to the same entity, scattering them in the embedding space. This "identity crisis" hinders the model's ability to accurately identify entity boundaries and types, especially in noisy or informal text.
The NerCo Framework
To address this, the authors introduce NerCo (Named entity recognition with Contrastive learning), a two-stage training paradigm designed to foster semantic cohesion within entity representations.
Stage 1: Semantic Class Guided Contrast
In the first stage, the model focuses on the broad semantic category of an entity (e.g., PER, ORG, GPE) rather than specific positional BIO tags. Using an InfoNCE loss, the model is trained to pull representations of tokens belonging to the same semantic class closer together while pushing tokens from different classes apart. This ensures that all tokens of a "Person" entity share a cohesive semantic core, regardless of whether they are at the beginning or in the middle of the name.
Stage 2: Token Label Supervised Fine-tuning
Once the encoder has established these cohesive semantic clusters, the model undergoes standard fine-tuning. This stage utilizes the traditional BIO tagging scheme and a Conditional Random Field (CRF) layer to teach the model positional nuances—identifying exactly where an entity starts and ends. By building on the strong semantic foundation from Stage 1, the model can more effectively discriminate between positional labels without losing the underlying entity identity.
Experimental Evaluation
NerCo was evaluated across four major Chinese NER benchmarks:
- OntoNotes 4.0: A large-scale news-domain corpus.
- MSRA: A standard news dataset from Microsoft Research Asia.
- Weibo NER: Noisy social media data containing informal mentions.
- Resume NER: A specialized dataset of executive resumes.
The framework uses the FLAT Transformer as its base encoder, requiring no additional data augmentation or complex architectural modifications.
Key Results and Insights
NerCo achieved state-of-the-art (SOTA) performance on all tested datasets. The most significant gains were observed on the Weibo NER dataset, where the F1-score improved by 4.11% over the baseline. This suggests that the cohesive representations learned via contrastive learning are particularly beneficial for noisy, small-scale datasets where traditional sequence labeling might struggle.
Furthermore, as illustrated in Figure 6, NerCo demonstrates superior stability when processing long sentences. While traditional models like BERT and FLAT show significant performance drops as sentence length increases, NerCo maintains a higher and more consistent F1-score. This robustness is attributed to the "anchoring" effect of cohesive semantic representations, which helps the model maintain entity context over longer sequences.
Conclusion
NerCo effectively mitigates the problem of entity representation segmentation by separating semantic clustering from positional classification. This two-stage approach leads to more distinct entity clusters and improved boundary detection. Future research may extend this framework to handle nested or discontinuous entities, further broadening its applicability in complex NLP tasks.
Summary updated