Multi-Head Adapter Routing for Data-Efficient Fine-Tuning

Lucas Caccia, Edoardo Maria Ponti, Lucas Liu, Matheus Pereira, Nicolas Le Roux, Alessandro Sordoni

Abstract

Multi-Head Adapter Routing (MHR) improves how large language models (LLMs) generalize to new tasks by decomposing parameter-efficient adapters into smaller, independently routed "heads." This granular control reduces task interference during multi-task training and provides a superior initialization for few-shot learning. MHR consistently outperforms existing methods like LoRA and Polytropon while maintaining high parameter efficiency.

Introduction

Fine-tuning massive language models for every new task is computationally prohibitive. Parameter-Efficient Fine-Tuning (PEFT) methods, such as LoRA, mitigate this by adding a small number of trainable parameters (adapters) to a frozen backbone. However, achieving robust cross-task generalization remains a challenge. Using a single adapter for all tasks often leads to negative transfer (interference), while using task-specific adapters prevents knowledge sharing.

Previous approaches like Polytropon (Poly) introduced an inventory of adapters with a routing function to select them. However, Poly's routing is "coarse," selecting entire adapters at once. This research introduces Multi-Head Routing (MHR) to provide more expressive, fine-grained control and investigates the underlying mechanisms that make routing effective.

Methods

Multi-Head Routing (MHR) draws inspiration from Multi-Head Attention by splitting adapter parameters into $h$ distinct "heads."

Figure 1: Overview of the Multi-Head Routing (MHR) architecture, showing how adapters are split into heads and routed based on task-specific weights.
  • Granular Routing: Instead of selecting a single adapter, MHR learns separate routing weights for every head. This allows the model to dynamically recombine "skills" from different modules at a much finer level of detail.
  • MHR-z (Routing-Only): In this variant, adapter modules are frozen after initial multi-task training, and only the minimal routing matrix is updated for new tasks, offering extreme parameter efficiency.
  • MHR-$\mu$ (Adapter Averaging): To test the necessity of active routing, the authors proposed averaging trained modules into a single adapter. This helps determine if routing is essential during inference or primarily serves as a better initialization strategy.

Experimental Setup

The approach was evaluated on two major cross-task generalization benchmarks:

  • T0 Dataset: 313 tasks spanning natural language inference, summarization, and more.
  • Super-Natural Instructions (SuperNI): A large-scale dataset with over 1,600 diverse tasks.
  • Backbone Models: Experiments were conducted across various scales, including T5-XL, T0-3B, and T0-11B.

Key Insights

  • Expressivity via Heads: Increasing the number of routing heads is significantly more effective than simply increasing the number of adapter modules. This multi-head structure allows for "piecewise linear" parameter combinations, enhancing model flexibility.
  • Gradient Alignment: A key discovery of this work is that MHR improves optimization by aligning the training gradients of different tasks. This reduces the "tug-of-war" interference typically seen in multi-task training, leading to more stable and effective learning.
  • Routing as Initialization: Surprisingly, active routing is not always required during final task adaptation. Averaging the pre-trained modules (MHR-$\mu$) often performs as well as active routing, suggesting that the primary benefit of routing is providing a powerful starting point for adaptation.

Key Results

Figure 5: Performance comparison on the T0 benchmark. MHR achieves higher accuracy than baselines like LoRA and Poly while maintaining parameter efficiency.
  • State-of-the-Art Performance: MHR outperformed Polytropon by 1.1% and standard LoRA by 3.1% on the T0 benchmark.
  • Efficiency Gains: MHR-z achieved competitive results while updating orders of magnitude fewer parameters than standard fine-tuning methods.
  • Zero-Shot Generalization: The MHR-$\mu$ averaging technique yielded gains of up to 3% absolute accuracy on the T0-11B model compared to baselines in zero-shot scenarios.
  • Scalability: The advantages of MHR were consistent across model scales, up to 11 billion parameters.

Limitations and Future Work

MHR's effectiveness depends on a diverse set of source tasks for initial training; its performance may degrade if the source task distribution is too narrow. Additionally, the multi-head logic is currently tailored to Transformer architectures. Future research could explore applying these gradient alignment and modular averaging principles to other architectures, such as Mixture-of-Experts (MoE) models.

Summary updated

Abstract

Parameter-efficient fine-tuning (PEFT) for cross-task generalization consists in pre-training adapters on a multi-task training set before few-shot adaptation to test tasks. Polytropon [Ponti et al., 2023] ($\texttt{Poly}$) jointly learns an inventory of adapters and a routing function that selects a (variable-size) subset of adapters for each task during both pre-training and few-shot adaptation. In this paper, we investigate the role that adapter routing plays in its success and design new variants based on our findings. First, we build on the intuition that finer-grained routing provides more expressivity. Hence, we propose $\texttt{MHR}$ (Multi-Head Routing), which combines $\textit{subsets}$ of adapter parameters and outperforms $\texttt{Poly}$ under a comparable parameter budget; by only fine-tuning the routing function and not the adapters ($\texttt{MHR}$-$z$), we achieve competitive performance with extreme parameter efficiency. Second, we find that $\texttt{Poly}$/$\texttt{MHR}$ performance is a result of better multi-task optimization, rather than modular inductive biases that facilitate adapter recombination and local adaptation, as previously hypothesized. In fact, we find that $\texttt{MHR}$ exhibits higher gradient alignment between tasks than any other method. Since this implies that routing is only crucial during multi-task pre-training, we propose $\texttt{MHR}$-$\mu$, which discards routing and fine-tunes the average of the pre-trained adapters during few-shot adaptation. This establishes $\texttt{MHR}$-$\mu$ as an effective method for single-adapter fine-tuning.
BibTeX
@inproceedings{caccia2023multi,
  author = {Lucas Caccia and Edoardo Maria Ponti and Lucas Liu and Matheus Pereira and Nicolas Le Roux and Alessandro Sordoni},
  title = {{Multi-Head Adapter Routing for Data-Efficient Fine-Tuning}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2023},
  url = {https://openreview.net/forum?id=Vn4XRQqjgn},
}
Figure