Robust Few-Shot Named Entity Recognition with Boundary Discrimination and Correlation Purification

Xiaojun Xue, Chunxia Zhang, Tianxiang Xu, Zhendong Niu

Introduction

Named Entity Recognition (NER) is a cornerstone of Natural Language Processing, essential for extracting structured information from unstructured text. While modern neural models excel in high-resource settings, Few-Shot NER aims to generalize to new domains using only a handful of examples. Despite its promise, few-shot NER remains vulnerable to adversarial perturbations. Specifically, models often rely on "surface features"—memorizing specific words rather than understanding semantic context. This makes them "brittle" when faced with synonym substitution attacks, where an entity name is replaced by a semantically similar word that the model has not encountered in its limited training set.

Adversarial Vulnerability in Few-Shot NER

The primary security flaw addressed in this work is the synonym substitution attack. In a cross-domain transfer learning scenario, a model trained on a source domain (e.g., general news) might fail to recognize an entity in a target domain (e.g., historical texts) if the specific wording changes, even if the context remains clear to a human reader.

Illustration of cross-domain transfer and adversarial synonym substitution attacks in NER.

As shown in the figure above, an attacker can swap "Bibliotheca" for "Edifice." While both refer to a building or library in context, the substitution can cause a standard model to misclassify the entity or fail to detect its boundaries entirely. This performance gap highlights the need for models that prioritize deep semantic relationships over rote memorization.

The BDCP Framework

We propose BDCP (Boundary Discrimination and Correlation Purification), a novel framework designed to bolster the robustness of few-shot NER. BDCP decouples the NER task into two specialized stages: span detection and entity typing, each enhanced with mechanisms to resist adversarial noise.

The BDCP framework architecture, consisting of Boundary Discriminative Span Detection and Correlation Purified Entity Typing.

Boundary Discriminative Span Detection

The first stage focuses on identifying the boundaries of potential entities. Unlike standard token classification, BDCP uses an Entity Boundary Discriminative Module. This module maps token features into a space where each boundary type (e.g., Start, Inside, End) is represented by a set of diverse components rather than a single vector.

  • Assignment Loss ($L_a$): This loss ensures that each token feature is correctly mapped to its most similar boundary component.
  • Diversity Loss ($L_d$): To prevent these components from converging into a single point (representation collapse), the diversity loss encourages them to remain distinct. This forces the model to learn a more generalized and robust representation of what constitutes an entity boundary, making it less sensitive to specific word choices.

Correlation Purified Entity Typing

Once a span is detected, the second stage classifies it into an entity type (e.g., Person, Organization). To handle the noise introduced by adversarial attacks, we apply the Information Bottleneck (IB) principle.

  • Correlation Facilitating ($L_p$): By maximizing the shared information between an entity and its surrounding context using InfoNCE loss, the model learns to rely on stable, generalizable semantic relationships.
  • Correlation Purification ($L_r$): This component uses KL divergence to minimize "interference information." It effectively filters out the specific, non-essential details of an adversarial example that might otherwise "fool" the network, leaving only the purified semantic signal necessary for accurate typing.

Experimental Results

The effectiveness of BDCP was tested on the Few-NERD and Cross-Dataset benchmarks against state-of-the-art baselines like the "Decomposed" model. Adversarial examples were generated using BERT-Attack to evaluate robustness.

Performance under Attack

BDCP consistently outperformed baselines, particularly in adversarial settings. On the Few-NERD (10-way 5-shot) benchmark, BDCP maintained an F1 score of 50.71 under attack, while the "Decomposed" baseline fell to 45.16. Similar trends were observed in cross-domain tests; for instance, in the News domain, BDCP achieved an F1 of 51.36 compared to the baseline's 42.87.

Visualization of Feature Space

Qualitative analysis using t-SNE plots further validates the framework. While baseline models produce overlapping and disorganized clusters when faced with adversarial data, BDCP creates tighter, more distinct clusters for different entity types. This visual evidence confirms that the "purification" process successfully separates the core semantic features from adversarial noise.

Conclusion

The BDCP framework provides a robust solution for few-shot NER by addressing the inherent brittleness of current models. By combining diverse boundary discrimination with information-theoretic correlation purification, BDCP ensures that models focus on invariant semantic signals rather than superficial word patterns. Future research will focus on extending these defenses to other adversarial modalities, such as character-level perturbations, and exploring joint optimization strategies for the two-stage architecture.

Summary updated

Abstract

Few-shot named entity recognition (NER) aims to recognize novel named entities in low-resource domains utilizing existing knowledge. However, the present few-shot NER models assume that the labeled data are all clean without noise or outliers, and there are few works focusing on the robustness of the cross-domain transfer learning ability to textual adversarial attacks in Few-shot NER. In this work, we comprehensively explore and assess the robustness of few-shot NER models under textual adversarial attack scenario, and found the vulnerability of existing few-shot NER models. Furthermore, we propose a robust two-stage few-shot NER method with Boundary Discrimination and Correlation Purification (BDCP). Specifically, in the span detection stage, the entity boundary discriminative module is introduced to provide a highly distinguishing boundary representation space to detect entity spans. In the entity typing stage, the correlations between entities and contexts are purified by minimizing the interference information and facilitating correlation generalization to alleviate the perturbations caused by textual adversarial attacks. In addition, we construct adversarial examples for few-shot NER based on public datasets Few-NERD and Cross-Dataset. Comprehensive evaluations on those two groups of few-shot NER datasets containing adversarial examples demonstrate the robustness and superiority of the proposed method.
BibTeX
@inproceedings{xue2024robust,
  author = {Xiaojun Xue and Chunxia Zhang and Tianxiang Xu and Zhendong Niu},
  title = {{Robust Few-Shot Named Entity Recognition with Boundary Discrimination and Correlation Purification}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2024},
  doi = {10.1609/aaai.v38i17.29904},
}
Figure