A Differentiable Language Model Adversarial Attack on Text Classifiers

Ivan Fursov, Alexey Zaytsev, Pavel Burnyshev, Ekaterina Dmitrieva, Nikita Klyuchnikov, Andrey Kravchenko, Ekaterina Artemova, Evgeny Burnaev

Abstract

Robustness of huge Transformer-based models for natural language processing is an important issue due to their capabilities and wide adoption. One way to understand and improve robustness of these models is an exploration of an adversarial attack scenario: check if a small perturbation of an input can fool a model. Due to the discrete nature of textual data, gradient-based adversarial methods, widely used in computer vision, are not applicable per se. The standard strategy to overcome this issue is to develop token-level transformations, which do not take the whole sentence into account. In this paper, we propose a new black-box sentence-level attack. Our method fine-tunes a pre-trained language model to generate adversarial examples. A proposed differentiable loss function depends on a substitute classifier score and an approximate edit distance computed via a deep learning model. We show that the proposed attack outperforms competitors on a diverse set of NLP problems for both computed metrics and human evaluation. Moreover, due to the usage of the fine-tuned language model, the generated adversarial examples are hard to detect, thus current models are not robust. Hence, it is difficult to defend from the proposed attack, which is not the case for other attacks.

Introduction

Adversarial attacks—small, intentional perturbations designed to fool machine learning models—are a cornerstone of robustness research. In computer vision, these attacks are facilitated by the continuous nature of pixel data, allowing for direct gradient-based optimization. Natural language, however, is discrete; words cannot be modified by infinitesimal increments. This fundamental difference has historically limited NLP adversarial research to rule-based or heuristic-driven "token-level" swaps.

Existing methods, such as synonym replacement or character-level noise, often rely on greedy search algorithms that are computationally expensive and frequently produce "clunky" or ungrammatical sentences. Moreover, because these attacks are often predictable, they can be mitigated by simple defenses like spell-checkers or adversarial training. We address these limitations with DILMA (DIfferentiable Language Model Attack), a framework that brings the power of gradient-descent to the discrete world of NLP by treating adversarial generation as a context-aware generative task.

Methodology

The DILMA framework centers on a fine-tuned Masked Language Model (MLM), such as BERT, which acts as the generator. Rather than searching for individual word swaps, we update the weights of the MLM to "steer" its output toward adversarial examples that satisfy specific constraints.

DILMA Architecture: The framework uses a Transformer generator, a Gumbel-Softmax sampler, and a differentiable loss function incorporating a surrogate classifier and Deep Levenshtein distance.

Differentiable Sampling and Distance

A major hurdle in gradient-based text attacks is the non-differentiable step of selecting a word from a vocabulary. DILMA overcomes this using the Straight-Through (ST) Gumbel-Softmax estimator, which provides a differentiable approximation of the sampling process.

Furthermore, to ensure the adversarial sentence remains semantically similar to the original, we must minimize the edit distance. Since standard metrics like Word Error Rate (WER) are not differentiable, we employ a Deep Levenshtein (DL) model—a neural network trained specifically to approximate edit distance. This model provides the smooth gradients necessary to keep the generator's output close to the source text.

Loss Function

The optimization process is driven by a differentiable loss function that balances two competing goals:

  1. Adversarial Success: Minimizing the probability of the correct class as predicted by a substitute LSTM classifier (which mimics the target black-box model).
  2. Stealthiness: Minimizing the approximate edit distance (via the DL model) to ensure the perturbation is minimal and contextually appropriate.

Experimental Results

We evaluated DILMA on four diverse benchmarks: AG News (AG), Dialogue State Tracking Challenge (DSTC), SST-2, and Rotten Tomatoes (RT). The primary metric used is the Normalised Accuracy Drop (NAD), which measures the reduction in classifier accuracy relative to the amount of change introduced to the text.

Quantitative comparison of adversarial attacks using the NAD metric across different datasets. The table highlights DILMA's superior performance in both standard and defended scenarios.

As shown in Figure 4, DILMA (particularly the "with DL" variant) consistently achieves the highest NAD scores. On the DSTC dataset, DILMA reached a NAD of 0.241, significantly outperforming the popular PWWS attack, which scored 0.142. Similar trends were observed in SST-2 and AG News.

Resistance to Defense

Perhaps the most significant finding is DILMA's effectiveness against defended models. Traditional attacks like HotFlip often see their success rates plummet when the target model is hardened via adversarial training. In contrast, DILMA remains potent. For example, on the SST-2 dataset, while other attacks were rendered nearly ineffective, DILMA maintained a NAD of 0.172 even after defense.

Human Evaluation and Stealth

Because DILMA uses a Transformer-based generator, the resulting adversarial sentences are more contextually coherent than those produced by simple word-swap heuristics. In human evaluation trials, crowd workers were able to correctly classify DILMA's adversarial examples approximately 70% of the time. This indicates that while the machine learning model was successfully deceived, the human-perceivable meaning of the sentence remained intact, confirming the attack's stealth and semantic preservation.

Conclusion

DILMA represents a significant advancement in textual adversarial attacks by making the process fully differentiable. By leveraging fine-tuned language models and neural approximations of edit distance, we can generate context-aware perturbations that are both highly effective and difficult to defend against. Future research will focus on extending this framework to more complex NLP tasks such as machine translation and natural language inference.

Summary updated

Abstract

Robustness of huge Transformer-based models for natural language processing is an important issue due to their capabilities and wide adoption. One way to understand and improve robustness of these models is an exploration of an adversarial attack scenario: check if a small perturbation of an input can fool a model. Due to the discrete nature of textual data, gradient-based adversarial methods, widely used in computer vision, are not applicable per~se. The standard strategy to overcome this issue is to develop token-level transformations, which do not take the whole sentence into account. In this paper, we propose a new black-box sentence-level attack. Our method fine-tunes a pre-trained language model to generate adversarial examples. A proposed differentiable loss function depends on a substitute classifier score and an approximate edit distance computed via a deep learning model. We show that the proposed attack outperforms competitors on a diverse set of NLP problems for both computed metrics and human evaluation. Moreover, due to the usage of the fine-tuned language model, the generated adversarial examples are hard to detect, thus current models are not robust. Hence, it is difficult to defend from the proposed attack, which is not the case for other attacks.
BibTeX
@inproceedings{fursov2022differentiable,
  author = {Ivan Fursov and Alexey Zaytsev and Pavel Burnyshev and Ekaterina Dmitrieva and Nikita Klyuchnikov and Andrey Kravchenko and Ekaterina Artemova and Evgeny Burnaev},
  title = {{A Differentiable Language Model Adversarial Attack on Text Classifiers}},
  booktitle = {IEEE Access},
  year = {2022},
  url = {https://openreview.net/forum?id=3OM6fkljcC},
}
Figure