TL;DR
This paper introduces Partial Embedding Matrix Adaptation (PEMA), a novel method to significantly reduce memory usage during language model fine-tuning by only updating the embeddings for the subset of vocabulary actually used in a given task. This approach achieves substantial memory savings without sacrificing downstream performance, making fine-tuning more accessible.
What Problem Does This Paper Address?
Fine-tuning large language models (LLMs) is computationally expensive, largely due to the massive embedding matrix that represents the model's vocabulary. This matrix can account for a significant portion of the model's parameters, demanding substantial memory for gradients and optimizer states. While previous work has focused on reducing the number of trainable parameters in other parts of the model, the memory footprint of the embedding matrix itself has remained a bottleneck, hindering the democratization of NLP and limiting fine-tuning on resource-constrained hardware.
Methods
The core of the paper's contribution is Partial Embedding Matrix Adaptation (PEMA). The method works in two main stages:
- Before Fine-tuning:
- Identify Used Vocabulary: The authors first analyze the fine-tuning dataset to determine the specific subset of tokens (the "partial vocabulary") that are actually present.
- Create Partial Embedding Matrix: A new, smaller embedding matrix is constructed containing only the embeddings for these used tokens.
- Remap Tokens: The input data is then remapped to use indices corresponding to this new, smaller embedding matrix.
- After Fine-tuning:
- Merge Embeddings: The newly learned embeddings from the partial matrix are merged back into the original, full embedding matrix (which is stored on disk). This ensures that the model retains embeddings for the entire vocabulary, allowing for future use on tasks with different vocabularies.
Crucially, PEMA does not update the embeddings for unused tokens during backpropagation, thus achieving memory savings without impacting performance.
Datasets
The authors evaluate their approach on the following datasets:
- GLUE benchmark: This benchmark comprises a variety of tasks, including:
- CoLA (Linguistic Acceptability)
- MNLI (Natural Language Inference)
- MRPC (Paraphrasing/Sentence Similarity)
- QNLI (Natural Language Inference)
- QQP (Paraphrasing/Sentence Similarity)
- RTE (Natural Language Inference)
- SST-2 (Sentiment Analysis)
- STS-B (Sentence Similarity)
- WNLI (Natural Language Inference)
- XNLI: This dataset extends MNLI to 15 languages, used for evaluating multilingual models.
The paper notes that they use publicly available versions of these datasets from Hugging Face.
Key Insights and Ideas
- Vocabulary Underutilization: A significant finding is that many fine-tuning datasets only utilize a small fraction of the total vocabulary of large language models. This observation is the cornerstone of their proposed method.
- PEMA's Orthogonality: PEMA is presented as orthogonal to many existing parameter-efficient fine-tuning (PEFT) methods. This means it can be combined with techniques like adapters or LoRA to achieve even greater memory savings.
- No Performance Degradation: A key insight is that by only updating the embeddings for the used vocabulary, downstream task performance is not negatively impacted. Unused embeddings are simply not trained, preserving their original state.
- Multilingual Models Benefit Most: Multilingual models, which inherently have larger vocabularies, show the most substantial memory savings with PEMA.
Key Results
- Significant Memory Reduction: PEMA achieves substantial reductions in memory usage. For instance, the embedding matrix can be reduced by an average of 47.3% for BERT, 52.1% for RoBERTa, and 72.4% for DeBERTaV3 across GLUE tasks.
- Dataset-Specific Savings: Memory savings vary by dataset. For BERT, savings can be as high as 94.3% for the WNLI dataset but only 11.5% for QQP.
- Minimal Runtime Overhead: Applying PEMA incurs a negligible runtime cost. For most GLUE datasets, it takes less than half a second and less than 1% of the fine-tuning duration.
- Consistent Performance: The paper demonstrates that PEMA does not negatively impact downstream task performance, with results on validation sets being largely identical to baseline methods.
- Scalability: Memory savings increase with model size and vocabulary size. Multilingual models show particularly extreme savings.
Limitations and Future Work
- Runtime Cost for Dataset Analysis: The initial step of analyzing the fine-tuning dataset to determine vocabulary usage incurs a runtime cost. However, the authors state this cost is negligible and easily parallelizable.
- Dual-Use Concerns: The authors acknowledge the dual-use nature of LLMs and that improving efficiency could lower the barrier for misuse.
- Future Work: The authors suggest adapting their approach for the output embedding matrix to achieve further memory savings.
Summary updated