A Differentiable Language Model Adversarial Attack on Text Classifiers

Ivan Fursov, Alexey Zaytsev, Pavel Burnyshev, Ekaterina Dmitrieva, Nikita Klyuchnikov, Andrey Kravchenko, Ekaterina Artemova, Evgenia Komleva, Evgeny Burnaev

Adversarial attacks—small, intentional perturbations designed to fool machine learning models—are a staple of computer vision research. In the world of pixels, these attacks are "easy" because images exist in a continuous space where you can follow gradients to find the perfect, invisible tweak. But text is discrete. You can’t change a word by 0.01; you either swap it for another word or you don't. This discreteness has long made gradient-based attacks on NLP models a significant challenge.

In their paper, "A Differentiable Language Model Adversarial Attack on Text Classifiers," Fursov et al. introduce DILMA (DIfferentiable Language Model Attack), a new framework that brings the power of gradient-based optimization to the discrete world of natural language.

TL;DR

DILMA is a black-box adversarial attack that fine-tunes a pre-trained language model to generate sentences that fool classifiers while remaining semantically similar to the original. By using a differentiable approximation of edit distance and the Gumbel-Softmax trick, it outperforms traditional word-swap methods and remains effective even against models protected by adversarial training.

What Problem Does This Paper Address?

Most existing adversarial attacks in NLP rely on "token-level" transformations—swapping a word for a synonym or changing a character to a lookalike. These methods often use greedy search or random heuristics, which are computationally expensive and frequently produce "clunky" sentences that don't respect the overall context.

Furthermore, because these attacks are often rule-based (like simple spell-swaps), they are relatively easy to defend against using adversarial training or spell-checkers. The authors identify a gap: we need a way to perform sentence-level attacks that are:

  1. Differentiable: Allowing us to use powerful gradient-descent techniques.
  2. Context-Aware: Ensuring the generated adversarial sentence still makes sense to a human.
  3. Robust: Capable of bypassing common defense strategies.

Methods

The core of DILMA is a fine-tuned Masked Language Model (MLM), like BERT. Instead of just searching for a replacement word, the authors actually update the weights of the MLM to "steer" it toward generating adversarial examples.

Key technical innovations include:

  • The Gumbel-Softmax Trick: To pass gradients through the discrete step of picking a word from a vocabulary, the authors use the Gumbel-Softmax estimator. This creates a "differentiable sampler" that bridges the gap between continuous optimization and discrete text.
  • Deep Levenshtein (DL): Standard edit distance (how many changes are made to a sentence) is not differentiable. The authors trained a separate neural network to approximate the Word Error Rate (WER). This "Deep Levenshtein" model provides a smooth gradient that tells the generator how to keep the adversarial sentence close to the original.
  • Differentiable Loss Function: The loss function balances two goals:
  • Fooling the model: Minimizing the probability of the correct class (using a substitute LSTM classifier).
  • Staying stealthy: Minimizing the Deep Levenshtein distance between the original and the new sentence.

Datasets

The authors evaluated DILMA on four diverse NLP benchmarks:

  1. AG News (AG): A 4-class news topic classification dataset.
  2. Dialogue State Tracking Challenge (DSTC): A 6-class intent prediction dataset.
  3. Stanford Sentiment Treebank (SST-2): Binary sentiment analysis of movie reviews.
  4. Rotten Tomatoes (RT): Another binary sentiment dataset for movie reviews.

Key Insights and Ideas

  • Fine-tuning as Attack: Rather than treating the attack as a search problem (finding the best word to swap), DILMA treats it as a generative problem. By fine-tuning the MLM's parameters during the attack, the model learns the specific "direction" it needs to move to fool the target.
  • Holistic Context: Because DILMA uses a Transformer-based generator, the adversarial changes are context-aware. It doesn't just swap a word; it generates a version of the sentence that "flows" better than simple synonym replacement.
  • The Power of Differentiable Distance: Using a neural network to approximate edit distance is a clever workaround for the non-differentiability of discrete metrics. This allows the model to "know" exactly how much it can change a sentence before it becomes too different from the source.

Key Results

The authors used the Normalised Accuracy Drop (NAD) metric, which measures how much the classifier's accuracy falls relative to the amount of change made to the text.

  • Superior Performance: DILMA (with the DL term) consistently achieved the highest NAD. For example, on the DSTC dataset, DILMA reached a NAD of 0.241, significantly higher than the popular PWWS attack (0.142).
  • Resistance to Defense: This is the "killer app" for DILMA. When the target models were defended using adversarial training, most other attacks (like HotFlip) saw their effectiveness vanish. DILMA, however, remained potent. On SST-2, while other attacks dropped to near-zero effectiveness after defense, DILMA maintained a NAD of 0.172.
  • Human Evaluation: In a study with crowd workers, humans were still able to correctly classify DILMA's adversarial sentences ~70% of the time. This confirms that while the model was fooled, the human-perceivable meaning remained intact.

Limitations and Future Work

  • Substitute Model Dependency: DILMA relies on a substitute classifier (an LSTM) to estimate the target model's behavior. If the substitute model doesn't mimic the target well, the attack might fail.
  • Computational Overhead: Fine-tuning a language model for every attack iteration is more computationally intensive than simple rule-based word swapping.
  • Future Directions: The authors suggest exploring more complex substitute models and extending the framework to other NLP tasks beyond classification, such as Natural Language Inference (NLI) or machine translation.

Summary updated

Abstract

Transformer models play a crucial role in state of the art solutions to problems arising in the field of natural language processing (NLP). They have billions of parameters and are typically considered as black boxes. Robustness of huge Transformer-based models for NLP is an important question due to their wide adoption. One way to understand and improve robustness of these models is an exploration of an adversarial attack scenario: check if a small perturbation of an input invisible to a human eye can fool a model. Due to the discrete nature of textual data, gradient-based adversarial methods, widely used in computer vision, are not applicable per se. The standard strategy to overcome this issue is to develop token-level transformations, which do not take the whole sentence into account. The semantic meaning and grammatical correctness of the sentence are often lost in such approaches In this paper, we propose a new black-box sentence-level attack. Our method fine-tunes a pre-trained language model to generate adversarial examples. A proposed differentiable loss function depends on a substitute classifier score and an approximate edit distance computed via a deep learning model. We show that the proposed attack outperforms competitors on a diverse set of NLP problems for both computed metrics and human evaluation. Moreover, due to the usage of the fine-tuned language model, the generated adversarial examples are hard to detect, thus current models are not robust. Hence, it is difficult to defend from the proposed attack, which is not the case for others. Our attack demonstrates the highest decrease of classification accuracy on all datasets(on AG news: 0.95 without attack, 0.89 under SamplingFool attack, 0.82 under DILMA attack).
BibTeX
@inproceedings{fursov2022differentiable,
  author = {Ivan Fursov and Alexey Zaytsev and Pavel Burnyshev and Ekaterina Dmitrieva and Nikita Klyuchnikov and Andrey Kravchenko and Ekaterina Artemova and Evgenia Komleva and Evgeny Burnaev},
  title = {{A Differentiable Language Model Adversarial Attack on Text Classifiers}},
  booktitle = {IEEE Access},
  year = {2022},
  url = {https://openreview.net/forum?id=3OM6fkljcC},
}

Loading related content...

Figure