TL;DR
This paper introduces SeqAttack, a framework for generating adversarial attacks on Named Entity Recognition (NER) models. They demonstrate that character and word-level attacks are effective and that adversarial training can significantly improve NER model robustness with minimal performance degradation.
What Problem Does This Paper Address?
Named Entity Recognition (NER) is a crucial task in Natural Language Processing (NLP) for extracting structured information like names, locations, and organizations. While adversarial attacks have been extensively studied for text classification tasks, their effectiveness and impact on NER models remain less understood. Deep learning models, including those used for NER, are known to be vulnerable to adversarial examples – subtly modified inputs that cause misclassifications. This paper aims to bridge this gap by investigating how adversarial attacks perform against NER models and exploring adversarial training as a defense mechanism.
Methods
The core contribution is SeqAttack, a framework built upon TextAttack, specifically adapted for token classification tasks like NER. SeqAttack extends TextAttack by incorporating a goal function tailored for NER, which considers the classification of individual tokens within a sequence. The framework allows for the generation of adversarial examples by applying transformations (character, word, or sentence level) while adhering to constraints (e.g., maintaining semantic similarity).
The paper evaluates several attack strategies:
- DeepWordBug: Character-level attacks involving swaps, insertions, and deletions.
- BERT-Attack: Word-level attacks using BERT embeddings for synonym replacement and token insertion/merging.
- CLARE: Similar to BERT-Attack, focusing on replacements and insertions.
- SCPN: Sentence-level attacks generating paraphrases.
For defense, they employ adversarial training, specifically data augmentation. This involves using the attack strategies to generate adversarial examples from the training data and then retraining the NER model with this augmented dataset.
Datasets
The experiments were conducted on the CoNLL2003 dataset, a standard benchmark for Named Entity Recognition. The authors fine-tuned a BERTbase cased model on this dataset for their experiments. For adversarial training, they used a subset of the CoNLL2003 dataset.
Key Insights and Ideas
- NER Models are Vulnerable: Similar to text classification, NER models are susceptible to adversarial attacks, particularly at the character and word levels.
- Character-Level Attacks are Potent but Ungrammatical: DeepWordBug, a character-level attack, proved highly effective in fooling the model. However, the generated examples are often ungrammatical, making them less suitable for real-world adversarial training.
- Word-Level Attacks Offer a Better Trade-off: Word-level attacks like BERT-Attack and CLARE generate more fluent and semantically similar adversarial examples, making them more promising for adversarial training and data augmentation.
- Adversarial Training is Effective: Adversarial training, even with a small number of augmented samples, significantly improves the robustness of NER models against these attacks without substantially degrading performance on clean data.
- Attack Strategy Matters for Training: The effectiveness of adversarial training depends on the attack strategy used. More potent attacks (like DeepWordBug) require fewer samples to achieve robustness, but the resulting models might be less generalizable.
Key Results
- DeepWordBug-I5 was the most effective attack strategy, reducing the model's accuracy significantly.
- BERT-Attack and CLARE were less effective than DeepWordBug-I5 but produced more grammatically sound adversarial examples.
- Adversarial training with DeepWordBug-I5 on just 500 examples increased the after-attack accuracy by 21% with minimal impact on standard test metrics.
- Adversarial training with BERT-Attack on 500 examples increased after-attack accuracy by 6% and reduced the attack rate by 4%.
- The paper also introduces SeqAttack, a framework and a companion web application for generating and inspecting adversarial examples for NER.
Limitations and Future Work
- Sentence-Level Attacks: The paper found sentence-level attacks (like SCPN) to be less effective on the CoNLL2003 dataset, possibly due to the limited syntactic forms used by the paraphrasing method. Future work could explore more diverse paraphrasing methods.
- Attack on Entities: The authors suggest future work could attempt to apply word-level attacks directly to named entities themselves, which could improve efficiency and uncover more realistic adversarial examples.
- Low-Resource Scenarios: The authors highlight that adversarial training can be beneficial in low-resource scenarios, suggesting this as a potential direction for future research.
- Grammaticality vs. Effectiveness: A trade-off exists between the effectiveness of an attack and the grammaticality of the generated examples. Future research could focus on developing attacks that balance both.
Summary updated