Adversarial attacks—small, intentional perturbations designed to fool machine learning models—are a staple of computer vision research. In the world of pixels, these attacks are "easy" because images exist in a continuous space where you can follow gradients to find the perfect, invisible tweak. But text is discrete. You can’t change a word by 0.01; you either swap it for another word or you don't. This discreteness has long made gradient-based attacks on NLP models a significant challenge.
In their paper, "A Differentiable Language Model Adversarial Attack on Text Classifiers," Fursov et al. introduce DILMA (DIfferentiable Language Model Attack), a new framework that brings the power of gradient-based optimization to the discrete world of natural language.
TL;DR
DILMA is a black-box adversarial attack that fine-tunes a pre-trained language model to generate sentences that fool classifiers while remaining semantically similar to the original. By using a differentiable approximation of edit distance and the Gumbel-Softmax trick, it outperforms traditional word-swap methods and remains effective even against models protected by adversarial training.
What Problem Does This Paper Address?
Most existing adversarial attacks in NLP rely on "token-level" transformations—swapping a word for a synonym or changing a character to a lookalike. These methods often use greedy search or random heuristics, which are computationally expensive and frequently produce "clunky" sentences that don't respect the overall context.
Furthermore, because these attacks are often rule-based (like simple spell-swaps), they are relatively easy to defend against using adversarial training or spell-checkers. The authors identify a gap: we need a way to perform sentence-level attacks that are:
- Differentiable: Allowing us to use powerful gradient-descent techniques.
- Context-Aware: Ensuring the generated adversarial sentence still makes sense to a human.
- Robust: Capable of bypassing common defense strategies.
Methods
The core of DILMA is a fine-tuned Masked Language Model (MLM), like BERT. Instead of just searching for a replacement word, the authors actually update the weights of the MLM to "steer" it toward generating adversarial examples.
Key technical innovations include:
- The Gumbel-Softmax Trick: To pass gradients through the discrete step of picking a word from a vocabulary, the authors use the Gumbel-Softmax estimator. This creates a "differentiable sampler" that bridges the gap between continuous optimization and discrete text.
- Deep Levenshtein (DL): Standard edit distance (how many changes are made to a sentence) is not differentiable. The authors trained a separate neural network to approximate the Word Error Rate (WER). This "Deep Levenshtein" model provides a smooth gradient that tells the generator how to keep the adversarial sentence close to the original.
- Differentiable Loss Function: The loss function balances two goals:
- Fooling the model: Minimizing the probability of the correct class (using a substitute LSTM classifier).
- Staying stealthy: Minimizing the Deep Levenshtein distance between the original and the new sentence.
Datasets
The authors evaluated DILMA on four diverse NLP benchmarks:
- AG News (AG): A 4-class news topic classification dataset.
- Dialogue State Tracking Challenge (DSTC): A 6-class intent prediction dataset.
- Stanford Sentiment Treebank (SST-2): Binary sentiment analysis of movie reviews.
- Rotten Tomatoes (RT): Another binary sentiment dataset for movie reviews.
Key Insights and Ideas
- Fine-tuning as Attack: Rather than treating the attack as a search problem (finding the best word to swap), DILMA treats it as a generative problem. By fine-tuning the MLM's parameters during the attack, the model learns the specific "direction" it needs to move to fool the target.
- Holistic Context: Because DILMA uses a Transformer-based generator, the adversarial changes are context-aware. It doesn't just swap a word; it generates a version of the sentence that "flows" better than simple synonym replacement.
- The Power of Differentiable Distance: Using a neural network to approximate edit distance is a clever workaround for the non-differentiability of discrete metrics. This allows the model to "know" exactly how much it can change a sentence before it becomes too different from the source.
Key Results
The authors used the Normalised Accuracy Drop (NAD) metric, which measures how much the classifier's accuracy falls relative to the amount of change made to the text.
- Superior Performance: DILMA (with the DL term) consistently achieved the highest NAD. For example, on the DSTC dataset, DILMA reached a NAD of 0.241, significantly higher than the popular PWWS attack (0.142).
- Resistance to Defense: This is the "killer app" for DILMA. When the target models were defended using adversarial training, most other attacks (like HotFlip) saw their effectiveness vanish. DILMA, however, remained potent. On SST-2, while other attacks dropped to near-zero effectiveness after defense, DILMA maintained a NAD of 0.172.
- Human Evaluation: In a study with crowd workers, humans were still able to correctly classify DILMA's adversarial sentences ~70% of the time. This confirms that while the model was fooled, the human-perceivable meaning remained intact.
Limitations and Future Work
- Substitute Model Dependency: DILMA relies on a substitute classifier (an LSTM) to estimate the target model's behavior. If the substitute model doesn't mimic the target well, the attack might fail.
- Computational Overhead: Fine-tuning a language model for every attack iteration is more computationally intensive than simple rule-based word swapping.
- Future Directions: The authors suggest exploring more complex substitute models and extending the framework to other NLP tasks beyond classification, such as Natural Language Inference (NLI) or machine translation.
Summary updated