Robust Few-Shot Named Entity Recognition with Boundary Discrimination and Correlation Purification

Xiaojun Xue, Chunxia Zhang, Tianxiang Xu, Zhendong Niu

TL;DR

Few-shot Named Entity Recognition (NER) models are highly vulnerable to subtle textual adversarial attacks like synonym substitution. This paper introduces BDCP, a robust two-stage framework that enhances entity boundary detection and purifies entity-context correlations using information theory. BDCP significantly boosts cross-domain robustness against adversarial attacks without requiring expensive data augmentation or adversarial training.

What Problem Does This Paper Address?

Few-shot NER aims to recognize novel named entities (like names, locations, or specialized terms) in low-resource target domains using only a handful of labeled examples. While effective on clean data, current state-of-the-art models suffer from a major flaw: they assume the world is free of noise and outliers.

In reality, minor adversarial perturbations—such as replacing a single word with its synonym—can completely derail these models. For instance, as shown in Figure 1, changing "Bibliotheca Alexandrina [library]" to "Edifice Alexandrina" tricks a standard model into classifying it as a [hotel].

Vulnerabilities of few-shot NER under synonym substitution attack

This vulnerability exists because models often rely on brittle, surface-level shortcuts and context preferences rather than learning generalized, transferable semantic correlations between entities and their surrounding text.

Methods

The authors propose BDCP (Boundary Discrimination and Correlation Purification), a robust two-stage few-shot NER framework that splits the task into span detection and entity typing:

Overview of the BDCP framework
  1. Boundary Discriminative Span Detection: To prevent boundary detection errors under attack, this module maps entity boundary classes (like BIOES tags) to multiple diverse sub-components in an angular margin-based feature space. It optimizes this space using two mutually complementary losses: an assignment loss ($L_a$) to align token representations to their closest boundary components, and a diversity loss ($L_d$) to prevent these components from collapsing into a single point.
  2. Correlation Purified Entity Typing: Once spans are detected, the model classifies their entity types. Grounded in the Information Bottleneck (IB) principle, this stage decomposes the mutual information between an entity and its context into "predictable" (generalized) and "specific" (noisy/adversarial) parts. It minimizes adversarial interference via a Kullback-Leibler (KL) divergence loss ($L_r$) and maximizes generalized features using an InfoNCE contrastive loss ($L_p$).

Datasets

The framework was evaluated on two primary few-shot NER benchmarks, artificially perturbed using BERT-Attack to generate synonym-substituted adversarial examples:

  • Few-NERD: A large-scale dataset evaluated across both INTRA (disjoint coarse-grained splits) and INTER (shared coarse-grained but disjoint fine-grained splits) tasks under 5-way 1-shot and 10-way 5-shot settings.
  • Cross-Dataset: A benchmark spanning four distinct domains: News (CoNLL-2003), Wiki (GUM), Social (WNUT-2017), and Mixed (Ontonotes).

Key Insights and Ideas

  • The Shortcut Trap: Standard few-shot models don't truly master cross-domain transfer; they memorize entity-context co-occurrence patterns. Synonym substitution breaks these superficial shortcuts, leading to catastrophic performance drops.
  • Diversity Defends Boundaries: Forcing the span detector to map boundary classes to multiple distinct sub-components (rather than a single vector) creates a more flexible, generalized boundary representation space that resists adversarial shifting.
  • Information Bottleneck as a Filter: By explicitly minimizing the upper bound of entity-specific context perturbations while maximizing the lower bound of core semantic correlations, the model effectively "purifies" the text, filtering out adversarial noise during entity typing.

Key Results

  • Superior Adversarial Robustness: BDCP consistently outperformed all baseline models (including ProtoBERT, CONTAINER, and Decomposed) across both clean and adversarial settings. On the Few-NERD adversarial benchmark, BDCP improved the F1 score by up to 7.59% over the strong Decomposed baseline.
  • Zero-Overhead Defense: Unlike popular defense mechanisms like RockNER or RanMASK, BDCP achieves its robustness purely through architectural and loss-function design, introducing no additional computational overhead like data augmentation or adversarial training during inference.
  • Clearer Decision Boundaries: As visualized via t-SNE in Figure 3, BDCP creates significantly tighter clusters and clearer decision boundaries for unseen entity types under attack compared to traditional models, which produce highly scattered and overlapping representations.
Figure 3: t-SNE Visualization of span representations on Few-NERD INTER 5-way 1-shot query set that containing adversarial examples. The representations are obtained from Decomposed (Ma et al. 2022) model and our BDCP model, respectively. Different colors represent different entity types.

Limitations and Future Work

  • Error Propagation: Because BDCP relies on a strict two-stage pipeline, any mistakes made during the initial span detection stage will inevitably propagate to and degrade the performance of the subsequent entity typing stage.
  • Future Directions: The authors plan to explore incorporating explicit adversarial training directly into the two-stage pipeline to further bolster cross-domain transfer learning. They also suggest leveraging both clean and attacked examples simultaneously within a unified contrastive learning framework.

Summary updated

Abstract

Few-shot named entity recognition (NER) aims to recognize novel named entities in low-resource domains utilizing existing knowledge. However, the present few-shot NER models assume that the labeled data are all clean without noise or outliers, and there are few works focusing on the robustness of the cross-domain transfer learning ability to textual adversarial attacks in Few-shot NER. In this work, we comprehensively explore and assess the robustness of few-shot NER models under textual adversarial attack scenario, and found the vulnerability of existing few-shot NER models. Furthermore, we propose a robust two-stage few-shot NER method with Boundary Discrimination and Correlation Purification (BDCP). Specifically, in the span detection stage, the entity boundary discriminative module is introduced to provide a highly distinguishing boundary representation space to detect entity spans. In the entity typing stage, the correlations between entities and contexts are purified by minimizing the interference information and facilitating correlation generalization to alleviate the perturbations caused by textual adversarial attacks. In addition, we construct adversarial examples for few-shot NER based on public datasets Few-NERD and Cross-Dataset. Comprehensive evaluations on those two groups of few-shot NER datasets containing adversarial examples demonstrate the robustness and superiority of the proposed method.
BibTeX
@inproceedings{xue2024robust,
  author = {Xiaojun Xue and Chunxia Zhang and Tianxiang Xu and Zhendong Niu},
  title = {{Robust Few-Shot Named Entity Recognition with Boundary Discrimination and Correlation Purification}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2024},
  doi = {10.1609/aaai.v38i17.29904},
}
Figure