TL;DR
This paper investigates the robustness of modern Named Entity Recognition (NER) models against adversarial attacks. The authors introduce four novel, model-agnostic attacks that don't require white-box access, revealing that current NER models heavily rely on case information and underutilize contextual cues, making them surprisingly fragile.
What Problem Does This Paper Address?
Named Entity Recognition (NER) is a fundamental task in Natural Language Processing (NLP), crucial for tasks like information extraction, question answering, and sentiment analysis. While NER models have seen significant advancements, especially with deep learning, their real-world performance can be hampered by noisy or slightly altered text. This paper addresses the critical question of how resilient these state-of-the-art NER models are when faced with subtle, adversarial perturbations in the input text. The authors aim to uncover weaknesses that might not be apparent in standard evaluations, particularly focusing on how models handle variations in case and context, which are common in real-world data.
Methods
The paper proposes four model-agnostic adversarial attacks designed to evaluate NER models without needing access to their internal workings (i.e., white-box access). These attacks are categorized into two main types:
- Case-based Adversarial Attacks:
- Case Ablation: This attack removes all case information from the text, simulating scenarios with informal text or OCR errors. The goal is to see how much NER models rely solely on capitalization.
- Case Aberration: This attack randomly capitalizes a percentage of tokens in the text. This aims to trick models into misclassifying non-entities as named entities.
- Context-based Adversarial Attacks:
- Context Perturbation: This method involves replacing named entities with other named entities of different classes within their immediate surrounding text, while preserving syntactic structure and semantics. This tests how well models use local context beyond just recognizing known entity tokens.
- Context Alteration: This attack globally alters the context by randomly placing named entities in unrelated locations, often resulting in grammatically incorrect sentences. The hypothesis here is that models which truly understand context should perform worse when the context is nonsensical.
The authors emphasize that their methods are rule-based and do not require retraining or fine-tuning of pre-trained models, making them efficient for evaluation.
Datasets
The study utilizes five publicly available, general-domain datasets for evaluation:
- CoNLL-2003: A widely used dataset comprising news articles from the Reuters Corpus.
- WikiGold: Manually annotated English Wikipedia articles.
- FIRE NER 2013: Text crawled from Indian websites and Wikipedia.
- NIST IE-ER 1999: Development test data from the NIST 1999 IE-ER Evaluation.
- GMB 2.2: A corpus of public domain texts including news, stories, and jokes, with preliminary annotations refined by human experts and crowd-sourcing.
These datasets cover the common NER classes: LOCATION, PERSON, and ORGANIZATION.
Key Insights and Ideas
- Over-reliance on Case: The most striking finding is that NER models are heavily dependent on case information. Even when case information is removed or corrupted, performance drops significantly, suggesting that models often use capitalization as a primary cue rather than a supplementary one.
- Underutilization of Context: While models are designed to leverage context, the context-based attacks reveal that they often fail to do so effectively. Models tend to memorize common named entities and their typical contexts, rather than truly understanding the semantic and syntactic roles they play.
- Model-Agnostic Evaluation is Feasible: The paper demonstrates that robust adversarial evaluation of NER models can be achieved without white-box access, using simple, rule-based generation methods. This makes the approach accessible for a wider range of researchers.
- Context Alteration as a Measure of True Understanding: The "Context Alteration" attack, where context is made nonsensical, is proposed as a way to gauge a model's genuine contextual understanding. Models that perform worse under this attack are considered to have a better grasp of context, as they are disrupted by the lack of meaningful cues.
Key Results
The paper presents F1 scores for four NER models (Flair, spaCy, CoreNLP-s, DeepPavlov) across the five datasets under various adversarial conditions.
- Case Ablation: Performance drops significantly across all models, with some models experiencing F1 score reductions of over 70%. This highlights the strong reliance on case.
- Case Aberration: Similar to ablation, case aberration also leads to substantial performance degradation, though generally slightly less severe than complete ablation.
- Context Perturbation: Despite mechanisms for context incorporation, models show large performance drops. Flair, which uses character embeddings, tends to perform better than others, suggesting it captures local context more effectively.
- Context Alteration: All models show performance drops, with Flair again demonstrating the largest drops, indicating it is more susceptible to global context disruption. This aligns with the hypothesis that models truly understanding context should be more affected by its absence or nonsensical nature.
For instance, under Case Ablation, the F1 score for Flair on the CONLL dataset drops from 0.92 to 0.37, a reduction of 52.03%. Similar drastic drops are observed for other models and datasets.
Limitations and Future Work
- Limitations: The authors acknowledge that mapping different tagging schemes (IOB, IOBES) to a unified IO scheme can cause their evaluation results to differ from officially reported scores. They also note that for the "Context Alteration" attack, low F1 scores could be due to a model being generally poor, not just its inability to capture global context, making absolute F1 scores less conclusive in this specific scenario.
- Future Work: The paper suggests that the proposed adversarial evaluation methods can be used to assess the generalizability of NER models. Further work is warranted to improve the resilience of NER models, particularly in their utilization of contextual information. The authors also hint at the potential for these methods to study the effect of unseen named entities with the same label.
Summary updated