WARP: Word-level Adversarial ReProgramming

Karen Hambardzumyan, Hrant Khachatrian, Jonathan May

Abstract

Transfer learning from pretrained language models recently became the dominant approach for solving many NLP tasks. A common approach to transfer learning for multiple tasks that maximize parameter sharing trains one or more task-specific layers on top of the language model. In this paper, we present an alternative approach based on adversarial reprogramming, which extends earlier work on automatic prompt generation. Adversarial reprogramming attempts to learn task-specific word embeddings that, when concatenated to the input text, instruct the language model to solve the specified task. Using up to 25K trainable parameters per task, this approach outperforms all existing methods with up to 25M trainable parameters on the public leaderboard of the GLUE benchmark. Our method, initialized with task-specific human-readable prompts, also works in a few-shot setting, outperforming GPT-3 on two SuperGLUE tasks with just 32 training samples.

Introduction

In Natural Language Processing (NLP), the standard paradigm has long been "pre-train then fine-tune." While effective, updating millions of parameters for every new task is computationally expensive and logistically challenging for large-scale deployments. Researchers from YerevaNN and USC propose WARP (Word-level Adversarial ReProgramming), a technique that "reprograms" frozen language models by learning a small set of task-specific input embeddings rather than modifying the model itself.

The WARP Framework

WARP draws inspiration from adversarial reprogramming in computer vision, where small perturbations to input pixels can redirect a classifier's behavior. In the context of NLP, WARP inserts "virtual tokens"—trainable vectors in the embedding space—into the input sequence.

Extracted figure from page 3 of WARP: Word-level Adversarial ReProgramming

The method consists of three core components:

  1. Prompt Embeddings: Trainable vectors inserted at specific positions (prefix, infix, or suffix) in the input.
  2. Frozen Backbone: A pre-trained model (e.g., RoBERTa or ALBERT) whose weights remain completely unchanged during training.
  3. The Verbalizer: A small linear layer that maps the model's Masked Language Model (MLM) head output at a [MASK] position into specific task classes (e.g., mapping specific tokens to "entailment" or "neutral").

By training only the prompt embeddings and the verbalizer, WARP utilizes fewer than 25,000 parameters—roughly 0.01% of the parameters in a standard RoBERTa-Large model.

Experimental Evaluation

The researchers evaluated WARP on the GLUE benchmark and few-shot subsets of SuperGLUE (FewGLUE).

Performance and Efficiency

WARP achieved an average score of 81.6 on the GLUE leaderboard, outperforming larger adapter-based models while using significantly fewer trainable parameters. Furthermore, the frozen backbone allows for high-throughput inference; a single model instance can handle multiple tasks simultaneously by simply swapping the tiny prompt vectors in the input batch.

Few-Shot Learning

A key strength of WARP is its performance in low-data regimes. When initialized with human-readable prompts (e.g., "The movie was [MASK]"), WARP provides a significant head start over training from scratch. In tests using only 32 training samples, WARP (built on ALBERT-xxlarge) outperformed GPT-3 on the CB and RTE tasks.

Few-shot performance comparison on the RTE task. WARP initialized with manual prompts (WARP_init) significantly outperforms standard fine-tuning and discrete prompting methods in low-data regimes.

Conclusion and Limitations

WARP demonstrates that massive language models can be effectively repurposed without internal weight modifications. However, the learned prompts are often uninterpretable to humans, appearing as "gibberish" when mapped back to the nearest dictionary words. While WARP offers extreme parameter efficiency and competitive few-shot performance, a performance gap remains compared to full fine-tuning of the latest state-of-the-art architectures like DeBERTa. Future work may extend this reprogramming approach to generative tasks beyond classification and regression.

Summary updated

Abstract

Transfer learning from pretrained language models recently became the dominant approach for solving many NLP tasks. A common approach to transfer learning for multiple tasks that maximize parameter sharing trains one or more task-specific layers on top of the language model. In this paper, we present an alternative approach based on adversarial reprogramming, which extends earlier work on automatic prompt generation. Adversarial reprogramming attempts to learn task-specific word embeddings that, when concatenated to the input text, instruct the language model to solve the specified task. Using up to 25K trainable parameters per task, this approach outperforms all existing methods with up to 25M trainable parameters on the public leaderboard of the GLUE benchmark. Our method, initialized with task-specific human-readable prompts, also works in a few-shot setting, outperforming GPT-3 on two SuperGLUE tasks with just 32 training samples.
BibTeX
@inproceedings{hambardzumyan2021warp,
  author = {Karen Hambardzumyan and Hrant Khachatrian and Jonathan May},
  title = {{WARP: Word-level Adversarial ReProgramming}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2021},
  doi = {10.18653/v1/2021.acl-long.381},
}
Figure