Generating Black-Box Adversarial Examples for Text Classifiers Using a Deep Reinforced Model

Prashanth Vijayaraghavan, Deb Roy

Adversarial attacks pose a significant threat to the reliability of machine learning models. While well-studied in computer vision, generating adversarial examples for text is challenging due to the discrete and fragile nature of language. This paper introduces the Adversarial Example Generator (AEG), a reinforcement learning-based model designed for black-box attacks. By employing a hybrid word-character architecture, AEG generates perturbations that effectively fool state-of-the-art classifiers while maintaining semantic integrity and human readability.

Introduction

The vulnerability of deep neural networks to adversarial examples—inputs specifically crafted to cause misclassification—is a critical concern. In natural language processing (NLP), creating such examples is difficult because small changes can drastically alter meaning or grammatical correctness. Most existing methods rely on "white-box" access to model gradients. However, "black-box" scenarios, where only the model's output probabilities are accessible, represent a more realistic threat. This research presents the AEG, a model that learns to exploit these vulnerabilities through a reinforced, hybrid approach.

The AEG Architecture

The core of the proposed method is the Adversarial Example Generator (AEG), which utilizes an encoder-decoder framework optimized via reinforcement learning.

Encoder: Computing Salient Information

The encoder is designed to process input text at both the word and character levels. It uses an attention mechanism to identify "salient" parts of the sentence—those most influential to the target classifier's decision. This attention mechanism effectively serves as a substitute network, mimicking the target model's internal logic without requiring direct access to its parameters.

Encoder architecture of the AEG model, processing both word and character level information.

Decoder: Hybrid Generation Process

The decoder features a two-level structure consisting of a word-level GRU and a character-level GRU. This hybrid design allows the AEG to choose the most effective perturbation strategy for a given context: either replacing a word with a semantically similar synonym (paraphrasing) or introducing subtle, human-readable typos (misspelling).

Decoder architecture of the AEG model, illustrating the dual-level generation of words and characters.

Training via Reinforcement Learning

To fine-tune the generator for adversarial success, the authors employ Self-Critical Sequence Training (SCST), a policy gradient method. The model is rewarded based on a multi-objective function:

  • Adversarial Reward: Success in inducing a misclassification by the target model.
  • Semantic Similarity: Preservation of the original meaning, measured by a separate deep matching model.
  • Lexical Similarity: Minimization of changes to the original text, measured by an approximate Levenshtein distance.

Experimental Results

The AEG was evaluated against word-based CNNs (IMDB sentiment analysis) and character-based CNNs (AG’s News categorization). The results demonstrate a significant drop in target model performance:

  • IMDB Dataset: The target model's accuracy plummeted from 89.95% to 18.5%, a reduction of nearly 80%.
  • AG’s News Dataset: Accuracy fell from 89.11% to 24.8%, a reduction of over 72%.

Crucially, human evaluation confirmed the quality of the generated text. In blind tests, humans correctly classified the sentiment of adversarial examples 94.6% of the time, proving that the perturbations successfully fooled the machine without confusing the human reader.

Key Insights and Future Work

The success of the AEG highlights the power of hybridity; combining word-level and character-level perturbations is far more effective than either strategy alone. While character-based models (like CNN-Char) showed some inherent robustness to misspellings, they still succumbed to the AEG's multi-pronged strategy.

Future work will focus on adversarial training—using these generated examples to retrain classifiers, making them more robust against such sophisticated perturbations in the future.

Summary updated

Abstract

Recently, generating adversarial examples has become an important means of measuring robustness of a deep learning model. Adversarial examples help us identify the susceptibilities of the model and further counter those vulnerabilities by applying adversarial training techniques. In natural language domain, small perturbations in the form of misspellings or paraphrases can drastically change the semantics of the text. We propose a reinforcement learning based approach towards generating adversarial examples in black-box settings. We demonstrate that our method is able to fool well-trained models for (a) IMDB sentiment classification task and (b) AG’s news corpus news categorization task with significantly high success rates. We find that the adversarial examples generated are semantics-preserving perturbations to the original text.
BibTeX
@inproceedings{vijayaraghavan2019generating,
  author = {Prashanth Vijayaraghavan and Deb Roy},
  title = {{Generating Black-Box Adversarial Examples for Text Classifiers Using a Deep Reinforced Model}},
  booktitle = {ECML/PKDD},
  year = {2019},
  url = {https://openreview.net/forum?id=0Dyv3aF37km},
}
Figure