TL;DR
This paper introduces "Ahead-of-Time P-Tuning" (AoT P-Tuning), a novel parameter-efficient fine-tuning method for large language models. AoT P-Tuning achieves competitive performance on downstream tasks while significantly reducing inference time and enabling efficient multi-task inference, addressing a key trade-off in existing methods.
What Problem Does This Paper Address?
The rapid growth in size of pre-trained language models (LMs) presents a challenge for efficient fine-tuning and deployment. While parameter-efficient fine-tuning methods aim to reduce computational costs by optimizing only a subset of parameters, they often introduce a trade-off between inference speed and the ability to perform multi-task inference (using a single model for multiple tasks). Some methods, like BitFit, offer zero inference overhead and multi-task capabilities but suffer from lower performance. Others, like P-Tuning and Adapters, can be competitive but incur inference overhead or have limitations for multi-task scenarios. This paper seeks to bridge this gap by proposing a method that is both parameter-efficient, fast for inference, and capable of multi-task inference.
Methods
The core innovation is Ahead-of-Time (AoT) P-Tuning. Unlike traditional P-Tuning which concatenates soft prompts to the input sequence, AoT P-Tuning injects input-dependent biases before each Transformer layer. This "ahead-of-time" modification allows the model to process sequences without increasing their length, thus avoiding the inference overhead associated with longer sequences.
To make the training of these biases tractable, two reparametrization strategies are proposed:
- Kronecker AoT P-Tuning: The bias matrix is factorized using a Kronecker product, significantly reducing the number of trainable parameters.
- FC AoT P-Tuning: The bias matrix is learned by passing the pre-trained embedding matrix through a learnable Fully Connected (FC) network. This approach leverages existing knowledge in the embeddings.
The paper primarily uses the FC AoT P-Tuning for experiments.
Datasets
The proposed method was evaluated on the following standard NLP benchmarks:
- GLUE: A collection of diverse natural language understanding tasks.
- SuperGLUE: A more challenging benchmark designed to test the robustness of models.
The experiments utilized the following pre-trained language models as backbones:
- RoBERTa-Base
- RoBERTa-Large
- DeBERTa-XL
Key Insights and Ideas
- Shifting the computation: The key insight is that by adding input-dependent biases before each Transformer layer, rather than concatenating prompts to the input, the sequence length remains unchanged. This is crucial for avoiding inference overhead.
- Trade-off resolution: AoT P-Tuning effectively resolves the common trade-off between inference speed and multi-task capability. It offers near-zero overhead and supports multi-task inference, outperforming methods like BitFit in performance while matching or exceeding others in efficiency.
- Reparameterization is key: The proposed reparametrization methods (Kronecker and FC) are essential for making the training of these new parameters feasible, especially for large models where direct optimization of biases for every layer would be prohibitive.
- FC AoT P-Tuning's advantage: The FC AoT P-Tuning method, which utilizes pre-trained embeddings, generally performs better than the Kronecker factorization approach, suggesting the benefit of leveraging existing model knowledge.
Key Results
- Performance: AoT P-Tuning (specifically FC AoT P-Tuning) achieves performance comparable to or better than established parameter-efficient methods like P-Tuning v1/v2, LoRA, and Adapters on GLUE and SuperGLUE benchmarks. It significantly outperforms BitFit.
- Inference Speed: AoT P-Tuning introduces negligible inference overhead, especially for larger models and longer sequences, often performing as fast as or even slightly faster than plain fine-tuning. In contrast, other methods like LoRA and Adapters can incur noticeable overhead.
- Multi-task Capability: AoT P-Tuning inherently supports multi-task inference with a single backbone model, a significant advantage for real-world applications.
- Parameter Efficiency: The reparametrization techniques ensure that the number of trainable parameters remains small, making it a truly parameter-efficient method.
Limitations and Future Work
- Hyperparameter Sensitivity: The paper notes that AoT P-Tuning can be sensitive to the number of hyperparameter assignments, though it generally outperforms P-Tuning v2 with fewer assignments.
- Alternative Reparameterizations: The authors suggest that further exploration of alternative reparametrizations for the bias matrix P could potentially enhance performance and reduce the need for extensive hyperparameter tuning.
- Architectural Modifications: Implementing various architectural modifications could further improve AoT P-Tuning's performance and efficiency.
Summary updated