Introduction
Named Entity Recognition (NER) is a cornerstone of Natural Language Processing, essential for extracting structured information from unstructured text. While modern neural models excel in high-resource settings, Few-Shot NER aims to generalize to new domains using only a handful of examples. Despite its promise, few-shot NER remains vulnerable to adversarial perturbations. Specifically, models often rely on "surface features"—memorizing specific words rather than understanding semantic context. This makes them "brittle" when faced with synonym substitution attacks, where an entity name is replaced by a semantically similar word that the model has not encountered in its limited training set.
Adversarial Vulnerability in Few-Shot NER
The primary security flaw addressed in this work is the synonym substitution attack. In a cross-domain transfer learning scenario, a model trained on a source domain (e.g., general news) might fail to recognize an entity in a target domain (e.g., historical texts) if the specific wording changes, even if the context remains clear to a human reader.
As shown in the figure above, an attacker can swap "Bibliotheca" for "Edifice." While both refer to a building or library in context, the substitution can cause a standard model to misclassify the entity or fail to detect its boundaries entirely. This performance gap highlights the need for models that prioritize deep semantic relationships over rote memorization.
The BDCP Framework
We propose BDCP (Boundary Discrimination and Correlation Purification), a novel framework designed to bolster the robustness of few-shot NER. BDCP decouples the NER task into two specialized stages: span detection and entity typing, each enhanced with mechanisms to resist adversarial noise.
Boundary Discriminative Span Detection
The first stage focuses on identifying the boundaries of potential entities. Unlike standard token classification, BDCP uses an Entity Boundary Discriminative Module. This module maps token features into a space where each boundary type (e.g., Start, Inside, End) is represented by a set of diverse components rather than a single vector.
- Assignment Loss ($L_a$): This loss ensures that each token feature is correctly mapped to its most similar boundary component.
- Diversity Loss ($L_d$): To prevent these components from converging into a single point (representation collapse), the diversity loss encourages them to remain distinct. This forces the model to learn a more generalized and robust representation of what constitutes an entity boundary, making it less sensitive to specific word choices.
Correlation Purified Entity Typing
Once a span is detected, the second stage classifies it into an entity type (e.g., Person, Organization). To handle the noise introduced by adversarial attacks, we apply the Information Bottleneck (IB) principle.
- Correlation Facilitating ($L_p$): By maximizing the shared information between an entity and its surrounding context using InfoNCE loss, the model learns to rely on stable, generalizable semantic relationships.
- Correlation Purification ($L_r$): This component uses KL divergence to minimize "interference information." It effectively filters out the specific, non-essential details of an adversarial example that might otherwise "fool" the network, leaving only the purified semantic signal necessary for accurate typing.
Experimental Results
The effectiveness of BDCP was tested on the Few-NERD and Cross-Dataset benchmarks against state-of-the-art baselines like the "Decomposed" model. Adversarial examples were generated using BERT-Attack to evaluate robustness.
Performance under Attack
BDCP consistently outperformed baselines, particularly in adversarial settings. On the Few-NERD (10-way 5-shot) benchmark, BDCP maintained an F1 score of 50.71 under attack, while the "Decomposed" baseline fell to 45.16. Similar trends were observed in cross-domain tests; for instance, in the News domain, BDCP achieved an F1 of 51.36 compared to the baseline's 42.87.
Visualization of Feature Space
Qualitative analysis using t-SNE plots further validates the framework. While baseline models produce overlapping and disorganized clusters when faced with adversarial data, BDCP creates tighter, more distinct clusters for different entity types. This visual evidence confirms that the "purification" process successfully separates the core semantic features from adversarial noise.
Conclusion
The BDCP framework provides a robust solution for few-shot NER by addressing the inherent brittleness of current models. By combining diverse boundary discrimination with information-theoretic correlation purification, BDCP ensures that models focus on invariant semantic signals rather than superficial word patterns. Future research will focus on extending these defenses to other adversarial modalities, such as character-level perturbations, and exploring joint optimization strategies for the two-stage architecture.
Summary updated