Adversarial Texts with Gradient Methods

Bo LiarXiv 2018 · arXiv

TL;DR

This paper tackles the challenge of generating adversarial text examples, which are subtly modified texts designed to fool machine learning models. The authors propose a novel framework that adapts successful gradient-based adversarial attack methods from image recognition to the text domain, overcoming the discrete nature of text inputs by operating in the continuous embedding space.

What Problem Does This Paper Address?

Adversarial examples have been extensively studied in computer vision, where adding small, imperceptible noise to images can cause models to misclassify them with high confidence. This research area has provided valuable insights into the robustness and limitations of deep learning models. However, applying these techniques to natural language processing (NLP) has been significantly more challenging due to two primary reasons:

  1. Discrete Input Space: Text is composed of discrete units (words or characters), unlike continuous pixel values in images. This makes it difficult to directly apply gradient-based methods that rely on accumulating small perturbations.
  2. Lack of Quality Metrics: Evaluating the "quality" of an adversarial text is not as straightforward as with images, where metrics like L2 or L-infinity norms are common. For text, a good adversarial example should ideally be semantically similar to the original and only slightly altered.

This paper aims to bridge this gap by developing a general framework to generate high-quality adversarial texts using gradient-based attack methods.

Methods

The core contribution of this paper is a framework that adapts gradient-based adversarial attack methods from image domains to text. The key innovations are:

  • Operating in Embedding Space: Instead of directly perturbing discrete text inputs, the authors first embed the text into a continuous vector space. Gradient-based attacks are then applied to these embeddings. This allows for the accumulation of small perturbations, similar to image attacks.
  • Reconstruction via Nearest Neighbor Search: After generating adversarial embeddings, the perturbed vectors are mapped back to actual words using a nearest neighbor search. This process reconstructs the adversarial text. The authors highlight that this reconstruction acts as a "strong denoising process," helping to maintain the semantic coherence of the text.
  • Word Mover's Distance (WMD) for Quality Measurement: To quantify the quality of adversarial texts, the paper proposes using Word Mover's Distance (WMD). WMD measures the minimum "work" required to transform the word embeddings of one document into those of another, effectively capturing semantic similarity. This provides a scalable and meaningful metric for evaluating adversarial text generation.

The authors successfully integrate two popular gradient-based image attack methods into their framework: Fast Gradient Method (FGM) and DeepFool.

Datasets

The paper evaluates its framework on three standard text classification datasets:

  • IMDB Movie Reviews: A binary sentiment classification dataset with 25,000 training and 25,000 testing samples.
  • Reuters-2: A subset of the Reuters newswire dataset, specifically focusing on two highly populated categories ("acq" and "earn") to create a balanced binary classification task.
  • Reuters-5: Another subset of the Reuters dataset, focusing on five categories ("crude," "grain," "interest," "money-fx," and "trade"), also balanced for classification.

The authors note that the Reuters dataset is naturally highly unbalanced, and they preprocess it to create more manageable and balanced subsets for their experiments.

Key Insights and Ideas

  • Embedding Space is Key: The discrete nature of text is a major hurdle, but operating in the continuous embedding space provides a smooth manifold where gradient-based attacks can be effectively applied.
  • Reconstruction is Crucial: Simply perturbing embeddings isn't enough; the reconstruction step using nearest neighbors is vital for generating coherent adversarial texts. This step implicitly filters out noise that doesn't correspond to meaningful word changes.
  • WMD as a Semantic Metric: WMD proves to be a robust metric for assessing adversarial text quality, correlating well with human judgment of semantic similarity and the number of word changes.
  • DeepFool's Superiority: The DeepFool algorithm, known for finding minimal perturbations in image domains, also yields the highest quality adversarial texts in this NLP context, often altering only a single word to change the predicted label.

Key Results

  • The proposed framework successfully generates adversarial texts that alter the predicted label of the original text.
  • DeepFool, when adapted to the framework, can change the label by modifying just one word in many cases, demonstrating high-quality adversarial generation.
  • FGSM and FGVM were also integrated, with FGVM showing improvement over FGSM by considering gradient magnitudes rather than just signs.
  • The paper reports accuracy measurements on adversarial embeddings before and after the nearest neighbor reconstruction. For instance, with FGSM and a noise scaling factor of 0.35, the accuracy on adversarial embeddings was 0.1213, which improved to 0.6770 after reconstruction. Similar improvements were observed across different methods and datasets.

Limitations and Future Work

  • Computational Bottleneck: The nearest neighbor search for reconstruction can be computationally expensive, especially with large vocabulary sizes. The authors address this by using Approximate Nearest Neighbor (ANN) techniques.
  • Fixed Embedding Space: The current framework relies on pre-trained, general-purpose word embeddings (GloVe). Future work could explore using task-specific embeddings, which might improve reconstruction quality and speed.
  • Limited Attack Types: The paper focuses on gradient-based methods. Future directions include adapting optimization-based and model-based attack methods.
  • WMD Minimization: The authors suggest using WMD as a target for optimization and model-based attacks, aiming to generate adversarial texts that minimize WMD to the original text.
  • Paraphrasing and Insertion/Deletion: The framework could be extended to handle more complex text modifications like paraphrasing and insertion/deletion of words, where WMD would be particularly useful.

Summary updated

BibTeX
@misc{li2018adversarial,
  author = {Bo Li},
  title = {{Adversarial Texts with Gradient Methods}},
  year = {2018},
  eprint = {1801.07175},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1801.07175},
}
Figure