Exploring the Vulnerability of Natural Language Processing Models via Universal Adversarial Texts

Xinzhe Li, Ming Liu, Xingjun Ma, Longxiang GaoALTA 2021 · OpenReview

Abstract

Universal adversarial texts (UATs) refer to short pieces of text units that can largely affect the predictions of NLP models. Recent studies on universal adversarial attacks assume the accessibility of datasets for the task, which is not realistic. We propose two types of Data-Free Adjusted Gradient (DFAG) attacks to show that it is possible to generate effective UATs with only one arbitrary example which could be manually crafted. Based on the proposed DFAG attacks, this paper explores the vulnerability of commonly used NLP models in terms of two factors: network architectures and pre-trained embeddings. Our empirical studies on three text classification datasets reveal that: 1) CNN based models are more extremely vulnerable to UATs while self-attention models show the most robustness, 2) the vulnerability of CNN and LSTM models and robustness of self-attention models could be attributed to whether they rely on training data artifacts for their predictions, and 3) the pre-trained embeddings could expose vulnerability to both universal adversarial attack and the UAT transfer attack.

Introduction

Deep learning models for Natural Language Processing (NLP) are notoriously fragile. While standard adversarial attacks involve tweaking a specific sentence to fool a model, Universal Adversarial Texts (UATs) are significantly more dangerous. These are static "triggers"—short phrases that, when prepended to any input, can flip a model's prediction regardless of the original context.

Historically, generating these triggers required a large "proxy" dataset to calculate gradients. However, in real-world scenarios, attackers rarely have access to a victim's training data. This research introduces a "data-free" approach to generating UATs and investigates why certain architectures are more susceptible to these attacks than others.

Methodology: Data-Free Adjusted Gradient (DFAG)

The core contribution of this work is the Data-Free Adjusted Gradient (DFAG) attack. Instead of averaging gradients over a large batch of real data, DFAG requires only a single "arbitrary" example, which can be manually authored.

To mitigate the noise inherent in calculating a gradient from a single data point, the researchers employed two approximation techniques borrowed from model explainability literature:

  1. DFAG (Smooth): Based on SmoothGrad, this method adds Gaussian noise to the input's embedding multiple times and averages the resulting gradients to find a stable direction.
  2. DFAG (Integrated): Based on Integrated Gradients, this samples points along a path from a "blank" baseline to the actual example, aggregating gradients along the way.
The DFAG attack process uses a single arbitrary example and gradient smoothing to update the UAT.

Once a reliable gradient is established, the method uses a linear approximation to iteratively swap tokens in the trigger sequence to maximize the model's prediction error.

Empirical Results and Architecture Vulnerability

The effectiveness of DFAG was evaluated across three benchmarks: SST-2 (sentiment), Yelp (reviews), and AG-News (topic classification). The study compared CNN, LSTM, and Self-attention architectures using various pre-trained embeddings (GloVe, FastText, BERT).

Attack Success Rates (ASR) demonstrate that CNNs and LSTMs are significantly more vulnerable than self-attention models.

Key Findings:

  • The Data-Free Surprise: DFAG attacks achieved performance comparable to baseline attacks that utilized hundreds of data samples. This suggests that vulnerability to UATs is a fundamental property "baked into" the model's weights during training.
  • Architecture Sensitivity: CNNs and LSTMs proved highly vulnerable. On the SST-2 dataset using GloVe embeddings, the CNN model reached a 100% Attack Success Rate (ASR). Conversely, self-attention models (Transformers) demonstrated significantly higher robustness.
  • The Role of Embeddings: Pre-trained embeddings like GloVe or FastText increase vulnerability. Triggers found for one model often "transfer" to another model if they share the same underlying word vectors, creating a shared security risk.

Discussion: Artifacts vs. Context

The disparity in robustness between architectures can be attributed to how they process information. CNNs and LSTMs tend to fixate on specific "artifact" tokens—spurious correlations in the training data where a single word (e.g., "horrible") is treated as a definitive signal for a class, regardless of context. UATs exploit these shallow shortcuts.

In contrast, self-attention models create contextualized representations. Every word's meaning is filtered through its relationship with every other word in the sentence. This holistic processing makes it much harder for a single static trigger to hijack the entire model's prediction.

Conclusion

This research highlights a critical security gap in NLP: models can be compromised using almost no data. While BERT-based models remain more resistant due to their multi-head attention and sub-word tokenization, the high vulnerability of CNN and LSTM architectures suggests that developers must move toward more contextualized, robust modeling techniques to defend against universal adversarial threats.

Summary updated

Abstract

Universal adversarial texts (UATs) refer to short pieces of text units that can largely affect the predictions of NLP models. Recent studies on universal adversarial attacks assume the accessibility of datasets for the task, which is not realistic. We propose two types of Data-Free Adjusted Gradient (DFAG) attacks to show that it is possible to generate effective UATs with only one arbitrary example which could be manually crafted. Based on the proposed DFAG attacks, this paper explores the vulnerability of commonly used NLP models in terms of two factors: network architectures and pre-trained embeddings. Our empirical studies on three text classification datasets reveal that: 1) CNN based models are more extremely vulnerable to UATs while self-attention models show the most robustness, 2) the vulnerability of CNN and LSTM models and robustness of self-attention models could be attributed to whether they rely on training data artifacts for their predictions, and 3) the pre-trained embeddings could expose vulnerability to both universal adversarial attack and the UAT transfer attack.
BibTeX
@inproceedings{li2021exploring,
  author = {Xinzhe Li and Ming Liu and Xingjun Ma and Longxiang Gao},
  title = {{Exploring the Vulnerability of Natural Language Processing Models via Universal Adversarial Texts}},
  booktitle = {ALTA},
  year = {2021},
  url = {https://openreview.net/forum?id=0x6w3ne4L9},
}
Figure