WARP: Word-level Adversarial ReProgramming

Karen Hambardzumyan, Hrant Khachatrian, Jonathan May

Abstract

WARP (Word-level Adversarial ReProgramming) is a lightweight, parameter-efficient method designed to repurpose frozen, pre-trained language models for downstream tasks. By learning a small set of continuous prompt embeddings and a task-specific verbalizer, WARP enables models to perform new tasks without modifying internal weights. With as few as 25,000 trainable parameters, WARP outperforms models with 1,000x more parameters on the GLUE benchmark and demonstrates superior few-shot learning capabilities compared to massive models like GPT-3.

Introduction

The prevailing paradigm for adapting large language models (LLMs) to specific tasks is full fine-tuning, which requires updating and storing millions or billions of parameters for every new application. While effective, this approach is computationally expensive and storage-intensive. Alternative methods like Adapters improve efficiency but still necessitate architectural modifications. Manual prompting, while requiring no training, is notoriously sensitive to phrasing and typically requires extremely large models to be effective.

WARP addresses these challenges by asking whether a frozen, medium-sized language model can be "reprogrammed" to perform new tasks solely through optimized input tokens. This approach draws inspiration from adversarial reprogramming in computer vision, where minimal input perturbations can redirect a classifier's behavior.

Methodology

WARP introduces a framework that keeps the core transformer weights frozen while optimizing two small sets of parameters:

  1. Continuous Prompts: Instead of searching for discrete words, WARP learns new embedding vectors ($[P_1], [P_2], \dots$) that are concatenated with the input text. These vectors exist in the model's high-dimensional embedding space and are optimized via gradient descent.
  2. The Verbalizer: To map model outputs to specific classes (e.g., "Entailment" vs. "Neutral"), WARP utilizes the pre-trained Masked Language Modeling (MLM) head. It learns specific "verbalizer" embeddings ($[V_1], [V_2], \dots$) that represent the target output classes.
The WARP architecture utilizes a frozen Transformer core, optimizing only the input prompt embeddings and the output verbalizer through the MLM head.

By focusing optimization only on the input and output layers, the entire "body" of the transformer remains untouched, allowing a single model instance to serve multiple tasks simultaneously by simply swapping tiny prompt files.

Experimental Setup

The researchers evaluated WARP on two primary benchmarks:

  • GLUE (General Language Understanding Evaluation): Nine diverse NLP tasks, including sentiment analysis (SST-2) and natural language inference (MNLI, QNLI).
  • FewGLUE: A subset of SuperGLUE used for few-shot learning experiments, specifically the CB (CommitmentBank) and RTE (Recognizing Textual Entailment) tasks using only 32 training samples.

Results and Discussion

WARP achieved an average score of 81.6 on the GLUE leaderboard, outperforming BERT-base and TinyBERT models despite using fewer than 25,000 trainable parameters. This represents a significant gain in parameter efficiency.

In few-shot scenarios, WARP demonstrated remarkable robustness. When built on the ALBERT model, it outperformed the 175B-parameter GPT-3 on the CB and RTE tasks. A key factor in this success is initialization; starting the continuous prompts with embeddings of human-designed instructions provides a significant performance boost over random initialization.

Comparison of accuracy across different training instance counts, showing WARP's competitive performance against full fine-tuning and its superiority in low-data regimes.

Furthermore, WARP offers a 2.5x to 3x increase in training speed compared to standard fine-tuning, as gradients are only calculated for the input layer rather than the entire transformer stack.

Conclusion and Future Work

WARP proves that the input space of a language model is a powerful lever for behavioral control. While it currently faces a small "performance tax" compared to full fine-tuning on large datasets (81.6 vs 88.1 on GLUE), its efficiency and few-shot capabilities make it a compelling alternative for deploying LLMs at scale.

Future research will focus on improving the interpretability of the learned "gibberish" prompts and extending the adversarial reprogramming framework to more complex generative tasks such as summarization and translation.

Summary updated

BibTeX
@inproceedings{hambardzumyan2021warp,
  author = {Karen Hambardzumyan and Hrant Khachatrian and Jonathan May},
  title = {{WARP: Word-level Adversarial ReProgramming}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2021},
  doi = {10.18653/v1/2021.acl-long.381},
}
Figure