Efficient Fine-Tuning of Compressed Language Models with Learners

Danilo Vucetic, Mohammadreza Tayaranian, Maryam Ziaeefard, James J. Clark, Brett H. Meyer, Warren J. GrossarXiv 2022 · OpenReview

TL;DR

This paper introduces "Learner modules," a novel method for efficiently fine-tuning large language models (LLMs) like BERT. By cleverly adding small, trainable "learner" components and using a "priming" technique, they achieve faster convergence and significantly reduced computational costs compared to existing methods, without sacrificing performance on downstream tasks.

What Problem Does This Paper Address?

Fine-tuning massive pre-trained language models (PLMs) like BERT for specific tasks is computationally expensive, requiring significant memory, processing power, and time. While many techniques exist to make these models faster at inference (e.g., pruning, quantization), fewer address the challenges of efficiently training them for new tasks. This is a critical bottleneck for deploying LLMs on resource-constrained devices or for rapid adaptation. The core problem is the "double bind" of efficient fine-tuning: how to train efficiently (fewer parameters, less computation) while still training effectively (achieving high performance and quick convergence).

Methods

The paper proposes "Learner modules" and a "priming" strategy.

  • Learner Modules: These are small, trainable components added in parallel to each linear layer of a target model. They consist of two low-rank projection matrices. The key idea is that these small modules learn for the larger linear layer. Crucially, after training, the Learner modules can be "collapsed" back into the original model's weights, meaning they don't add any overhead during inference. This is a key differentiator from methods like adapters.
  • Priming: This technique involves initially training the entire multi-head attention mechanism alongside the Learner projections for a few epochs, while keeping the rest of the model frozen. After this "priming" phase, training switches to only the Learner projections. This strategy aims to leverage the quick convergence of attention mechanisms and then efficiently fine-tune the specific task using the Learner modules.

Datasets

The primary dataset used for evaluation is the GLUE benchmark, a collection of diverse natural language understanding tasks. Specific datasets mentioned include:

  • CoLA (Corpus of Linguistic Acceptability)
  • MNLI (Multi-Genre Natural Language Inference)
  • MRPC (Microsoft Research Paraphrase Corpus)
  • QNLI (Question-answering NLI)
  • QQP (Quora Question Pairs)
  • RTE (Recognizing Textual Entailment)
  • SST-2 (Stanford Sentiment Treebank)
  • WNLI (Winograd Schema Challenge) - noted as being too small for useful evaluation in this context.

The models were trained on DistilBERT, a compressed version of BERT.

Key Insights and Ideas

  • Learners offer a sweet spot: The paper argues that existing efficient fine-tuning methods often trade off convergence speed for parameter efficiency (e.g., adapters) or vice-versa (e.g., FAR). Learners, combined with priming, aim to achieve both.
  • Priming is crucial for convergence: The "priming" step, where attention layers are trained initially, seems to be the key to achieving fast convergence even with parameter-efficient methods. This suggests that leveraging the initial learning capacity of core model components is vital.
  • Collapsible modules are key for inference: The ability to collapse Learner modules back into the original weights means they don't incur any inference cost, unlike methods that permanently add parameters. This is a significant advantage for deployment.
  • Compressed models pose unique challenges: The paper highlights that methods effective on large models might not translate well to compressed models that inherently have less parameterization to exploit.

Key Results

  • Performance on par or better: Learner modules perform on par with or surpass baseline methods on the GLUE benchmark.
  • 7x fewer parameters: Compared to state-of-the-art methods like FAR, Learners train significantly fewer parameters (e.g., 5.96 million vs. 41.45 million on DistilBERT).
  • Faster fine-tuning: On the CoLA dataset, Learners fine-tune 20% faster than some baselines.
  • Lower resource utilization: Learners demonstrate significantly lower resource utilization in terms of memory and computation.
  • Learner size matters: Larger Learner modules (e.g., Learner256) generally yield better results, but even smaller ones with priming (Learner64 (p=2)) can outperform larger ones without priming.

Limitations and Future Work

  • Future work on convergence: The authors suggest exploring alternative methods to further increase convergence speed, such as distillation or pre-fine-tuning on out-of-distribution tasks.
  • Alternative models: They also propose investigating the effectiveness of Learners on models with even more limited parameterization, like MobileBERT.
  • Priming trade-off: While priming unlocks efficient training, it does involve a temporary increase in resource usage during the initial epochs. The paper acknowledges this as a "temporary trade-off of efficiency for fine-tuning efficacy."

Summary updated

Abstract

Fine-tuning BERT-based models is resource-intensive in memory, computation, and time. While many prior works aim to improve inference efficiency via compression techniques, e.g., pruning, these works do not explicitly address the computational challenges of training to downstream tasks. We introduce Learner modules and priming, novel methods for fine-tuning that exploit the overparameterization of pre-trained language models to gain benefits in convergence speed and resource utilization. Learner modules navigate the double bind of 1) training efficiently by fine-tuning a subset of parameters, and 2) training effectively by ensuring quick convergence and high metric scores. Our results on DistilBERT demonstrate that learners perform on par with or surpass the baselines. Learners train 7x fewer parameters than state-of-the-art methods on GLUE. On CoLA, learners fine-tune 20% faster, and have significantly lower resource utilization.
BibTeX
@misc{vucetic2022efficient,
  author = {Danilo Vucetic and Mohammadreza Tayaranian and Maryam Ziaeefard and James J. Clark and Brett H. Meyer and Warren J. Gross},
  title = {{Efficient Fine-Tuning of Compressed Language Models with Learners}},
  year = {2022},
  eprint = {2208.02070},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2208.02070},
}
Figure