NOTABLE: Transferable Backdoor Attacks Against Prompt-based NLP Models

Kai Mei, Zheng Li, Zhenting Wang, Yang Zhang, Shiqing Ma

Introduction

Prompt-based learning has emerged as a dominant paradigm in Natural Language Processing (NLP), allowing pre-trained language models (PLMs) to be adapted to specific downstream tasks with minimal fine-tuning. By reformulating classification tasks as "cloze" (fill-in-the-blank) problems—for example, classifying a movie review by asking the model to fill the mask in "I liked this movie. It was [MASK]."—researchers have achieved state-of-the-art performance across diverse domains.

However, this paradigm introduces new security vulnerabilities. While traditional backdoor attacks poison training data to force misclassification when a specific "trigger" is present, these attacks are often "brittle." They frequently fail if a downstream user alters the prompt template or retrains the model on a different dataset. This paper introduces NOTABLE, a novel backdoor attack designed to be both task-free and prompt-free, ensuring high transferability and persistence across various NLP applications.

The NOTABLE Framework

The core innovation of NOTABLE lies in its move away from simple embedding poisoning toward a more integrated architectural attack. The framework relies on three primary components:

1. Encoder-Level Injection

Unlike previous methods that focus solely on the word embedding layer, NOTABLE updates the weights of the transformer encoders. By targeting the encoder, the attack forces the model’s internal attention mechanism to create "shortcut connections" between the trigger token and the target output. This deep integration makes the backdoor significantly harder to "wash out" during downstream fine-tuning or task adaptation.

2. Adaptive Verbalizer

In prompt-based learning, a "verbalizer" maps the model's internal predictions (e.g., the word "great") to specific labels (e.g., "Positive"). NOTABLE utilizes an adaptive verbalizer that combines manually selected words (such as "Yes" or "True") with words discovered through an automated search process. These words serve as semantic anchors for the attack.

3. Semantic Binding

Rather than binding a trigger to a specific label—which might change or lose meaning between different tasks—NOTABLE binds triggers to these semantic anchors. Because most classification tasks rely on shared underlying concepts (e.g., Positive vs. Negative sentiment, or Entailment vs. Contradiction in logic), the backdoor transfers seamlessly across different datasets and task types.

Experimental Methodology

The researchers evaluated NOTABLE using a two-stage experimental design to test its transferability:

  • Shadow Poisoning: The initial backdoor was injected into the PLM using "shadow" datasets, specifically Yelp and SQuAD 2.0. These datasets were chosen to be distinct from the eventual downstream tasks to simulate a real-world scenario where an adversary provides a compromised pre-trained model to an unsuspecting user.
  • Downstream Evaluation: The attack's effectiveness was tested across six diverse NLP tasks, including sentiment analysis (SST-2, IMDB), toxicity detection (Twitter), and Natural Language Inference (BoolQ, RTE, CB).

Key Insights and Results

NOTABLE demonstrated exceptional performance across multiple model architectures, including BERT, DistilBERT, and RoBERTa. The findings highlight several critical aspects of the attack's potency:

  • High Success Rates: The attack achieved an Attack Success Rate (ASR) exceeding 90% across all six downstream datasets, often reaching 100%. This confirms that the semantic binding strategy effectively bridges the gap between different tasks.
  • Utility Preservation: The "Clean Accuracy" (performance on non-poisoned inputs) remained nearly identical to that of benign models. This lack of performance degradation makes the attack highly stealthy, as users would have no immediate reason to suspect the model has been compromised.
  • Prompt Independence: Because the attack targets the encoder's processing of input tokens rather than the specific structure of the prompt, it remains effective regardless of whether the user employs manual prompts, automated discrete prompts, or continuous "soft" prompts.
  • Stealthy Triggers: NOTABLE can utilize uncommon but natural-looking tokens (e.g., cf or mn) as triggers. This allows the attack to bypass common defenses that flag high-perplexity or nonsensical string insertions.

Robustness Against Defenses

The authors tested NOTABLE against three state-of-the-art backdoor defense mechanisms: ONION, RAP, and T-Miner. NOTABLE remained highly effective against these defenses. Its resilience stems from the fact that the triggers are integrated into the encoder's attention logic, rather than appearing as simple statistical anomalies in the input text that traditional filters are designed to catch.

Conclusion and Future Work

NOTABLE represents a significant advancement in the study of NLP vulnerabilities, demonstrating that prompt-based models can be compromised in a way that is both persistent and transferable. While the current study focuses on classification tasks, the success of semantic anchor binding suggests that similar vulnerabilities may exist in other domains.

Future research may explore the applicability of NOTABLE to generative tasks, such as summarization or translation, and investigate whether similar transferable attacks can be executed in computer vision or multi-modal models where prompt-based learning is increasingly adopted.

Summary updated

BibTeX
@inproceedings{mei2023notable,
  author = {Kai Mei and Zheng Li and Zhenting Wang and Yang Zhang and Shiqing Ma},
  title = {{NOTABLE: Transferable Backdoor Attacks Against Prompt-based NLP Models}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2023},
  doi = {10.18653/v1/2023.acl-long.867},
}
Figure