Abstract
In this work, we explore "prompt tuning", a simple yet effective mechanism for learning "soft prompts" to condition frozen language models to perform specific downstream tasks. Unlike the discrete text prompts used by GPT-3, soft prompts are learned through backpropagation and can be tuned to incorporate signal from any number of labeled examples. Our end-to-end learned approach outperforms GPT-3's "few-shot" learning by a large margin. More remarkably, through ablations on model size using T5, we show that prompt tuning becomes more competitive with scale: as models exceed billions of parameters, our method "closes the gap" and matches the strong performance of model tuning (where all model weights are tuned). This finding is especially relevant in that large models are costly to share and serve, and the ability to reuse one frozen model for multiple downstream tasks can ease this burden. Our method can be seen as a simplification of the recently proposed "prefix tuning" of Li and Liang (2021), and we provide a comparison to this and other similar approaches. Finally, we show that conditioning a frozen model with soft prompts confers benefits in robustness to domain transfer, as compared to full model tuning.
Introduction
The standard paradigm for adapting pre-trained language models like BERT or T5 to new tasks is "model tuning" (fine-tuning), where every weight in the model is updated. While effective, this approach creates a significant deployment challenge: serving 100 different tasks requires storing and serving 100 separate copies of a multi-billion parameter model.
Alternative methods like "prompt design" (used in GPT-3) keep the model frozen and use manual text instructions. However, prompt design is often finicky, requires human expertise to craft the "perfect" prompt, and generally underperforms compared to full fine-tuning. This paper introduces Prompt Tuning, asking: Can we achieve the performance of fine-tuning with the efficiency of a frozen model?
Methods
Prompt Tuning prepends a sequence of $k$ "soft prompt" tokens to the input. Unlike discrete text prompts, these are continuous vectors (embeddings) learned through standard backpropagation while the underlying model remains frozen.
Key technical details include:
- Soft Prompts: Continuous embeddings that do not correspond to real vocabulary words, updated via backpropagation.
- Frozen Backbone: The entire multi-billion parameter T5 model is frozen during training; only the small prompt embedding matrix is updated.
- LM Adaptation: The authors found that T5, originally trained on "span corruption," requires a small amount of additional training on a standard language modeling objective to become an effective prompt-based learner.
- Initialization: Prompts are initialized from random noise, sampled vocabulary words, or class labels (e.g., embeddings for "true"/"false").
Key Insights and Results
The primary breakthrough of this work is the discovery that scale is the great equalizer. At small model sizes (e.g., 200M parameters), prompt tuning is significantly less effective than full fine-tuning. However, as the model grows to the "XXL" size (11B parameters), the performance gap completely vanishes.
Extreme Parameter Efficiency
For an 11B parameter model, prompt tuning requires storing only about 20,000 parameters per task. This represents a reduction of five orders of magnitude compared to full fine-tuning. This efficiency also enables "prompt ensembling," where multiple prompts can be run on the same model in a single batch, making ensembling faster and cheaper.
Performance and Robustness
On the SuperGLUE benchmark, T5-XXL with prompt tuning achieved a score of 89.1, matching the 89.3 achieved by full model tuning. Furthermore, prompt tuning demonstrates superior resilience to Domain Shift. Because the core "generalist" weights are frozen, the model is less likely to overfit to specific training dataset quirks. In zero-shot domain transfer tests (e.g., training on SQuAD and evaluating on MRQA), prompt tuning actually outperformed full fine-tuning.
Limitations and Future Work
- Scale Requirement: The method requires significant scale (typically >1B parameters) to match fine-tuning performance.
- Interpretability: Learned prompts form semantic clusters but are not human-readable sentences.
- Pre-training Mismatch: Models trained with "fill-in-the-blank" objectives require a one-time "LM Adaptation" step to be ready for prompt-based control.
Conclusion
Prompt tuning provides a powerful, parameter-efficient way to adapt massive language models. By freezing the backbone and tuning only a tiny set of prompt vectors, it achieves parity with full fine-tuning at scale while offering better robustness and significantly lower deployment overhead.
Summary updated