Robust Few-Shot Named Entity Recognition with Boundary Discrimination and Correlation Purification

Xiaojun Xue, Chunxia Zhang, Tianxiang Xu, Zhendong Niu

Abstract

Few-shot named entity recognition (NER) aims to recognize novel named entities in low-resource domains utilizing existing knowledge. However, present models often assume clean data, showing significant vulnerability to textual adversarial attacks during cross-domain transfer. We propose BDCP (Boundary Discrimination and Correlation Purification), a two-stage method designed for robustness. In the span detection stage, an entity boundary discriminative module provides a highly distinguishing representation space. In the entity typing stage, correlations between entities and contexts are purified using the Information Bottleneck principle to filter out adversarial noise. Evaluations on adversarial versions of Few-NERD and Cross-Dataset demonstrate BDCP's superior robustness and performance.

Introduction: The Vulnerability of Few-Shot NER

Few-shot NER is critical for specialized domains where labeled data is scarce. However, existing models are surprisingly fragile when faced with "adversarial" tweaks like synonym substitutions.

Adversarial attacks in cross-domain NER illustrate how synonym substitution can lead to misclassification.

As shown in Figure 1, a model might correctly identify "Bibliotheca Alexandrina" as a library, but misclassify its synonym "Edifice Alexandrina" as a hotel. This "memorization trap" occurs because models often rely on surface-level features rather than the underlying semantic relationship between an entity and its context.

Proposed Method: The BDCP Framework

To address these vulnerabilities, we introduce BDCP, a two-stage framework that enhances both entity detection and classification.

The BDCP framework architecture, featuring Boundary Discriminative Span Detection and Correlation Purified Entity Typing.

1. Boundary Discriminative Span Detection

Instead of simple token-level tagging, this module creates a highly distinguishing representation space for entity boundaries. It uses multiple "components" for each boundary class to capture diverse patterns. By employing assignment and diversity losses, the model ensures that entity boundaries are clearly separated in the feature space, making it harder for adversarial perturbations to cause detection errors.

2. Correlation Purified Entity Typing

Once a span is detected, the model must categorize it (e.g., Person vs. Organization). Adversarial attacks introduce "interference" in the context. Drawing on the Information Bottleneck (IB) principle, BDCP explicitly minimizes "interference information" (noise unique to the adversarial example) while maximizing "predictable information"—the core correlation between the entity and its context that remains stable even if words are swapped.

Experimental Results

The BDCP model was evaluated on two major benchmarks: Few-NERD and Cross-Dataset (covering News, Wiki, Social, and Mixed domains). To test robustness, adversarial versions of these datasets were generated using synonym substitution.

Key findings include:

  • Superior Robustness: On adversarial data, BDCP consistently outperformed state-of-the-art baselines like RockNER and RanMASK. For instance, in the Few-NERD "INTRA 5-1" task, BDCP achieved an F1 score of 40.76, significantly higher than the previous best of ~36.16.
  • Improved Generalization: Even on clean data, BDCP showed performance gains, suggesting that "purifying" correlations helps the model learn more generalizable features.
  • Clustered Representations: Visual analysis of feature spaces confirmed that BDCP creates much tighter, more distinct clusters for different entity types compared to standard models, which often suffer from overlapping representations that lead to misclassification.

Conclusion

BDCP significantly improves the robustness of few-shot NER by strengthening boundary detection and purifying semantic signals. While the current two-stage approach is effective, future work will focus on mitigating "error propagation" between stages and exploring end-to-end adversarial training to further harden models against complex textual attacks.

Summary updated

Abstract

Few-shot named entity recognition (NER) aims to recognize novel named entities in low-resource domains utilizing existing knowledge. However, the present few-shot NER models assume that the labeled data are all clean without noise or outliers, and there are few works focusing on the robustness of the cross-domain transfer learning ability to textual adversarial attacks in Few-shot NER. In this work, we comprehensively explore and assess the robustness of few-shot NER models under textual adversarial attack scenario, and found the vulnerability of existing few-shot NER models. Furthermore, we propose a robust two-stage few-shot NER method with Boundary Discrimination and Correlation Purification (BDCP). Specifically, in the span detection stage, the entity boundary discriminative module is introduced to provide a highly distinguishing boundary representation space to detect entity spans. In the entity typing stage, the correlations between entities and contexts are purified by minimizing the interference information and facilitating correlation generalization to alleviate the perturbations caused by textual adversarial attacks. In addition, we construct adversarial examples for few-shot NER based on public datasets Few-NERD and Cross-Dataset. Comprehensive evaluations on those two groups of few-shot NER datasets containing adversarial examples demonstrate the robustness and superiority of the proposed method.
BibTeX
@inproceedings{xue2024robust,
  author = {Xiaojun Xue and Chunxia Zhang and Tianxiang Xu and Zhendong Niu},
  title = {{Robust Few-Shot Named Entity Recognition with Boundary Discrimination and Correlation Purification}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2024},
  doi = {10.1609/aaai.v38i17.29904},
}
Figure