Towards Stronger Adversarial Baselines Through Human-AI Collaboration

Wencong You, Daniel LowdACL 2022

TL;DR

This paper tackles the limitations of current adversarial attacks on NLP models, which often produce unnatural or ungrammatical text. They propose a novel human-AI collaboration framework that combines human intuition with automated attack algorithms to generate more effective and realistic adversarial examples, leading to better evaluation of NLP model robustness.

What Problem Does This Paper Address?

Natural Language Processing (NLP) systems are increasingly used for critical tasks like spam detection, abuse identification, and fake news filtering. Evaluating the robustness of these systems against adversarial attacks is crucial. Traditional methods rely on static test sets, which are insufficient. Prior work has explored manually crafted adversarial examples, but these are limited by human creativity and can be time-consuming. Automated attacks, while efficient, often generate unnatural or ungrammatical text, failing to reflect real-world adversarial scenarios. This paper addresses the gap by proposing a hybrid approach that leverages both human linguistic expertise and the efficiency of AI-driven attacks to create more sophisticated and realistic adversarial examples for evaluating NLP models.

Methods

The core of this paper is a novel human-AI collaboration framework designed to generate adversarial examples for NLP tasks. The framework integrates four attack methods:

  • Auto: A baseline where participants simply evaluate pre-generated adversarial examples from automated algorithms.
  • Manual: Participants manually craft adversarial examples without AI assistance, relying solely on their judgment and the target model's prediction feedback.
  • Select: A "gray-box" method where participants are provided with intermediate perturbation results from automated attacks, including keywords and potential substitutions. They can then select the best word substitutes or enter their own.
  • Saliency: Another "gray-box" method that provides participants with a dynamic saliency map, highlighting words the target model deems most important for its prediction. This allows humans to make more informed modifications.

The framework also incorporates a detailed evaluation process for the generated adversarial examples, considering grammar, plausibility, effort required, and helpfulness of the AI assistance.

Datasets

The paper uses the following datasets for evaluation:

  • SST-2: A sentiment analysis dataset.
  • Hatebase: An abuse comment dataset.

The target models used are BERT and ROBERTa, which are state-of-the-art models for these tasks.

Key Insights and Ideas

  • Human intuition + AI efficiency = Better Attacks: The fundamental idea is that humans excel at understanding language nuances and semantics, while AI is efficient at exploring vast search spaces. Combining these strengths leads to more effective adversarial examples than either approach alone.
  • Real-world attacks are hybrid: The paper notes that real-world attackers already use semi-automated techniques. This research aims to mimic and improve upon these sophisticated methods for better evaluation.
  • Automated attacks often lack realism: The paper highlights that current automated attacks frequently produce nonsensical or ungrammatical text, which doesn't accurately reflect how humans would try to fool NLP systems.
  • Gray-box interaction is key: The "Select" and "Saliency" methods, which provide humans with insights into the automated attack process, are hypothesized to be more effective than purely black-box approaches.

Key Results

The paper presents preliminary results from a pilot experiment where the authors themselves used the system. While a full user study is pending IRB approval, these initial findings suggest:

  • Human-AI collaboration shows promise: An example is given where PWWS (an automated attack) identified a key word to modify, but a human was then able to propose a better substitute that tricked the classifier while maintaining semantics.
  • Attack success rates: While automated attacks might have a higher success rate due to intensive search, human-AI interaction methods are expected to achieve comparable success rates with better quality examples. Manually crafted attacks without any AI assistance are shown to be less effective.
  • Grammar and Plausibility: Human-generated adversarial examples, especially those aided by AI (Manual and HITL methods), tend to have higher scores for grammar and plausibility compared to purely automated attacks.
  • Effort vs. Queries: The gray-box settings (Select and Saliency) are found to be easier to attack than black-box settings, providing participants with more insight and flexibility.

Limitations and Future Work

  • Pilot Study: The current results are based on a pilot experiment conducted by the authors themselves. A full user study with external participants is pending IRB approval.
  • Dataset Expansion: The authors plan to include the IMDB movie review dataset in future experiments, as its longer texts may offer more flexibility for participants.
  • Broader Collaboration: The paper aims to explore more effective human-AI collaboration methods and hopes to establish a new benchmark for HITL adversarial learning.
  • Robust NLP Models: The ultimate goal is to help researchers and practitioners better evaluate NLP model robustness and improve models by training on these sophisticated adversarial examples.

Summary updated

BibTeX
@inproceedings{you2022towards,
  author = {Wencong You and Daniel Lowd},
  title = {{Towards Stronger Adversarial Baselines Through Human-AI Collaboration}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2022},
}
Figure