TL;DR
This paper introduces PrAd, a novel parameter-efficient tuning (PET) method for decoder-only language models that significantly improves inference efficiency, especially in multi-task scenarios, without sacrificing performance. By strategically applying adapters only during the prefill stage of inference, PrAd achieves substantial reductions in management costs and latency, making large language models more practical for real-world deployment.
What Problem Does This Paper Address?
Fine-tuning large language models (LLMs) for various downstream NLP tasks is effective but computationally expensive, requiring separate models for each task. Parameter-efficient tuning (PET) methods aim to mitigate this by updating only a small subset of parameters. However, existing PET approaches have limitations: prompt-based methods can increase input length and are sensitive to initialization, while adapter-based methods can introduce significant inference latency and management overhead, especially in multi-task settings. This paper addresses the need for a PET method that offers both strong performance and high efficiency, particularly for decoder-only LLMs in practical, multi-task applications.
Methods
The core innovation of PrAd is its Prompt Adaptive Tuning approach. It leverages the observation that LLM inference can be decoupled into two phases: prefill (encoding the input) and decode (generating the output). PrAd applies trainable adapter modules exclusively during the prefill stage, transforming the input prompt. The decode model remains unchanged. This strategy allows PrAd to:
- Maintain original input lengths: Unlike prefix tuning, it doesn't increase the sequence length.
- Simplify initialization: Adapters can be initialized easily, similar to other adapter-based methods.
- Reduce management costs: Adapters are only needed for the prefill stage, halving management and memory overhead in decoupled deployments.
- Eliminate decoding latency: Crucially, PrAd introduces no additional inference latency during the decoding phase, even when serving multiple tasks concurrently.
PrAd is presented as a general framework that can encompass various prompt-based methods by choosing different adapter architectures ("A") for the prompt transformation. The paper explores both parallel and sequential adapter variants within the PrAd framework.
Datasets
The paper evaluates PrAd on a diverse set of NLU and NLG tasks using the following datasets:
- SST2: Sentence-level binary classification for sentiment analysis.
- MNLI: Natural Language Inference task.
- E2E: Table-to-text generation task.
- WebNLG: Table-to-text generation task.
- MT: German to English translation task.
- XSUM: English summarization task.
The models used for experiments include GPT-2 Small, GPT-2 Large, and LLaMA-7B.
Key Insights and Ideas
- Decoupling prefill and decode is key: The paper highlights that separating these inference phases is crucial for optimizing service performance and efficiency, and PrAd capitalizes on this by applying adapters only to the prefill stage.
- Adapters only in prefill = efficiency gain: The core insight is that by confining adapter usage to the prefill stage, PrAd achieves significant inference speedups and reduced management overhead without compromising performance.
- No decoding latency is a game-changer: The ability to avoid additional latency during the critical decoding phase, especially in multi-task settings, is a major advantage for real-world deployment.
- PrAd as a unifying framework: The paper suggests that many existing prompt-based methods can be viewed as special instances of PrAd, highlighting its generality.
- Model scale matters: While PrAd might match baselines on smaller models, its advantages become more pronounced and competitive on larger LLMs.
Key Results
- Performance: PrAd methods consistently achieve comparable or superior performance to strong baselines like PA and LoRA across various NLU and NLG tasks. For instance, on GPT-2 Large, PrAd-PA emerged as the top-performing method on SST-2, MNLI, and MT.
- Inference Efficiency:
- In single-task settings, PrAd-LoRA introduces no extra latency, while PrAd-PA has minimal impact, outperforming PA which can add significant latency.
- In multi-task scenarios, PrAd achieves substantially higher efficiency than PA and LoRA. Table 4 shows PrAd-PA achieving a speedup of up to 13.96x compared to LoRA in multi-task settings.
- Management Costs: PrAd reduces adapter management and memory costs by up to 50% due to adapters being used only in the prefill stage.
Limitations and Future Work
- Pre-token latency: The paper acknowledges that using adapters in the prefill phase does introduce some additional computation cost for generating the first token.
- Task-specific characteristics: While PrAd performs competitively, it may underperform state-of-the-art methods on highly specific tasks, depending on task characteristics.
- Hyperparameter sensitivity: The choice of hyperparameters for adapters can impact performance, and further research is needed to fully understand this impact.
- Decoder-only focus: The current work is limited to decoder-only models. The authors suggest that extending PrAd to other architectures with autoregressive decoding (like encoder-decoder models) is a promising direction for future work, though its efficacy remains an open question.
Summary updated