Abstract
Multi-Head Adapter Routing (MHR) improves how large language models (LLMs) generalize to new tasks by decomposing parameter-efficient adapters into smaller, independently routed "heads." This granular control reduces task interference during multi-task training and provides a superior initialization for few-shot learning. MHR consistently outperforms existing methods like LoRA and Polytropon while maintaining high parameter efficiency.
Introduction
Fine-tuning massive language models for every new task is computationally prohibitive. Parameter-Efficient Fine-Tuning (PEFT) methods, such as LoRA, mitigate this by adding a small number of trainable parameters (adapters) to a frozen backbone. However, achieving robust cross-task generalization remains a challenge. Using a single adapter for all tasks often leads to negative transfer (interference), while using task-specific adapters prevents knowledge sharing.
Previous approaches like Polytropon (Poly) introduced an inventory of adapters with a routing function to select them. However, Poly's routing is "coarse," selecting entire adapters at once. This research introduces Multi-Head Routing (MHR) to provide more expressive, fine-grained control and investigates the underlying mechanisms that make routing effective.
Methods
Multi-Head Routing (MHR) draws inspiration from Multi-Head Attention by splitting adapter parameters into $h$ distinct "heads."
- Granular Routing: Instead of selecting a single adapter, MHR learns separate routing weights for every head. This allows the model to dynamically recombine "skills" from different modules at a much finer level of detail.
- MHR-z (Routing-Only): In this variant, adapter modules are frozen after initial multi-task training, and only the minimal routing matrix is updated for new tasks, offering extreme parameter efficiency.
- MHR-$\mu$ (Adapter Averaging): To test the necessity of active routing, the authors proposed averaging trained modules into a single adapter. This helps determine if routing is essential during inference or primarily serves as a better initialization strategy.
Experimental Setup
The approach was evaluated on two major cross-task generalization benchmarks:
- T0 Dataset: 313 tasks spanning natural language inference, summarization, and more.
- Super-Natural Instructions (SuperNI): A large-scale dataset with over 1,600 diverse tasks.
- Backbone Models: Experiments were conducted across various scales, including T5-XL, T0-3B, and T0-11B.
Key Insights
- Expressivity via Heads: Increasing the number of routing heads is significantly more effective than simply increasing the number of adapter modules. This multi-head structure allows for "piecewise linear" parameter combinations, enhancing model flexibility.
- Gradient Alignment: A key discovery of this work is that MHR improves optimization by aligning the training gradients of different tasks. This reduces the "tug-of-war" interference typically seen in multi-task training, leading to more stable and effective learning.
- Routing as Initialization: Surprisingly, active routing is not always required during final task adaptation. Averaging the pre-trained modules (MHR-$\mu$) often performs as well as active routing, suggesting that the primary benefit of routing is providing a powerful starting point for adaptation.
Key Results
- State-of-the-Art Performance: MHR outperformed Polytropon by 1.1% and standard LoRA by 3.1% on the T0 benchmark.
- Efficiency Gains: MHR-z achieved competitive results while updating orders of magnitude fewer parameters than standard fine-tuning methods.
- Zero-Shot Generalization: The MHR-$\mu$ averaging technique yielded gains of up to 3% absolute accuracy on the T0-11B model compared to baselines in zero-shot scenarios.
- Scalability: The advantages of MHR were consistent across model scales, up to 11 billion parameters.
Limitations and Future Work
MHR's effectiveness depends on a diverse set of source tasks for initial training; its performance may degrade if the source task distribution is too narrow. Additionally, the multi-head logic is currently tailored to Transformer architectures. Future research could explore applying these gradient alignment and modular averaging principles to other architectures, such as Mixture-of-Experts (MoE) models.
Summary updated