SPoT: Better Frozen Model Adaptation through Soft Prompt Transfer

Tu Vu, Brian Lester, Noah Constant, Rami Al-Rfou, Daniel Cer

TL;DR

This paper introduces SPoT (Soft Prompt Transfer), a novel method that significantly improves parameter-efficient prompt tuning for large language models. SPoT achieves this by transferring learned prompts from source tasks to initialize prompts for target tasks, matching or exceeding full model fine-tuning performance on SuperGLUE while using drastically fewer parameters.

What Problem Does This Paper Address?

The explosion of massive pre-trained language models (like GPT-3) has led to remarkable performance gains across NLP tasks. However, fine-tuning these models for each specific downstream task is computationally prohibitive, requiring significant memory and storage for each task-specific model. Parameter-efficient methods like Prompt Tuning offer a solution by learning only a small set of "soft prompt" parameters, keeping the large language model frozen. While effective, standard Prompt Tuning still lags behind full fine-tuning, especially for smaller models. This paper addresses the gap by proposing a transfer learning approach for prompt tuning, aiming to boost performance and efficiency.

Methods

The core innovation is SPoT: Soft Prompt Transfer. Instead of training a prompt from scratch for each target task, SPoT first trains a prompt on one or more "source" tasks. This learned source prompt is then used to initialize the prompt for the target task. This initialization acts as a warm-start, guiding the prompt learning process for the new task.

The authors explore two main flavors of SPoT:

  1. Generic SPoT: A single prompt is learned on a mixture of source tasks and then used to initialize prompts for all target tasks.
  2. Targeted SPoT: A more sophisticated approach where task embeddings are derived from learned prompts. These embeddings are used to find the most similar source tasks to a novel target task, and the corresponding learned source prompt is used for initialization. This involves a retrieval mechanism to select the best source prompts.

They compare SPoT against baselines like standard Prompt Tuning (PROMPTTUNING) and full Model Tuning (MODELTUNING).

Datasets

The experiments utilize a wide array of datasets from established NLP benchmarks:

  • GLUE: A collection of diverse NLP tasks.
  • SUPERGLUE: A more challenging benchmark designed to test a broader range of language understanding capabilities.
  • C4 (Colossal Clean Crawled Corpus): Used for unsupervised learning tasks.
  • MNLI (Multi-Genre Natural Language Inference): A large-scale dataset for NLI.
  • SQuAD (Stanford Question Answering Dataset): A popular dataset for question answering.

The paper also lists numerous other datasets categorized by task type (NLI, sentiment analysis, QA, commonsense reasoning, etc.) used in their extensive transferability studies.

Key Insights and Ideas

  • Transfer Learning is Key for Prompt Tuning: The most significant insight is that learned prompts from related tasks can provide a powerful initialization, dramatically improving the performance and stability of prompt tuning.
  • SPoT Closes the Performance Gap: SPoT effectively bridges the performance gap between parameter-efficient methods and full model fine-tuning, especially for smaller model sizes.
  • Task Embeddings Capture Semantic Relationships: Interpreting learned prompts as task embeddings reveals a meaningful semantic space of NLP tasks, where similar tasks cluster together. This is a novel way to understand task relationships.
  • Predicting Transferability is Possible: The task embedding approach allows for predicting which source tasks are most likely to benefit a novel target task, reducing the need for exhaustive search.
  • Scale Isn't Everything (for Prompt Tuning): While larger models generally perform better, SPoT demonstrates that effective transfer learning can make prompt tuning competitive even without massive model scaling.

Key Results

  • Performance Boost: SPoT provides significant improvements over standard Prompt Tuning, achieving +4.4 and +10.1 point accuracy gains on GLUE and SUPERGLUE respectively with T5 BASE.
  • Competitive with Model Tuning: Across all model sizes, SPoT matches or outperforms full Model Tuning on SUPERGLUE. At the XXL model size (11B parameters), SPoT even surpasses MULTI-TASKMODELTUNING.
  • Parameter Efficiency: SPoT achieves this performance while using up to 27,000x fewer task-specific parameters than full Model Tuning.
  • Task Transferability: A large-scale study across 26 NLP tasks showed that many tasks can benefit from prompt transfer, with some transfers yielding over 50% relative error reduction (e.g., MNLI → CB).
  • Efficient Retrieval: The targeted SPoT approach can eliminate 69% of the source task search space while retaining 90% of the best-case quality gain.
  • SUPERGLUE Performance: The SPOT XXL submission achieved a score of 89.2 on the SUPERGLUE leaderboard, outperforming other parameter-efficient methods and nearly matching fully fine-tuned T5 XXL.

Limitations and Future Work

  • Task Similarity Factors: While task embedding similarity is a strong predictor, other factors like data size, task difficulty, and domain similarity also influence transferability. The paper notes that their task embeddings don't capture all such factors.
  • Extending to Other Methods: The authors suggest exploring whether SPoT's transfer learning paradigm can be extended to other parameter-efficient adaptation methods.
  • Interpreting Soft Prompts: They hypothesize that soft prompts could be interpreted as natural language instructions, which warrants further investigation.
  • Broader Task Embedding Exploration: Future work could explore alternative methods for deriving task embeddings beyond using fixed prompt checkpoints.

Summary updated

Abstract

There has been growing interest in parameter-efficient methods to apply pre-trained language models to downstream tasks. Building on the Prompt Tuning approach of Lester et al. (2021), which learns task-specific soft prompts to condition a frozen pre-trained model to perform different tasks, we propose a novel prompt-based transfer learning approach called SPoT: Soft Prompt Transfer. SPoT first learns a prompt on one or more source tasks and then uses it to initialize the prompt for a target task. We show that SPoT significantly boosts the performance of Prompt Tuning across many tasks. More remarkably, across all model sizes, SPoT matches or outperforms standard Model Tuning (which fine-tunes all model parameters) on the SuperGLUE benchmark, while using up to 27,000x fewer task-specific parameters. To understand where SPoT is most effective, we conduct a large-scale study on task transferability with 26 NLP tasks in 160 combinations, and demonstrate that many tasks can benefit each other via prompt transfer. Finally, we propose an efficient retrieval approach that interprets task prompts as task embeddings to identify similar tasks and predict the most transferable source tasks for a novel target task.
BibTeX
@inproceedings{vu2022spot,
  author = {Tu Vu and Brian Lester and Noah Constant and Rami Al-Rfou and Daniel Cer},
  title = {{SPoT: Better Frozen Model Adaptation through Soft Prompt Transfer}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2022},
  url = {https://openreview.net/forum?id=AeBF2gyFQ7w},
}
Figure