Abstract
The fine-tuning of pre-trained language models has resulted in the widespread availability of task-specific models. Model merging offers an efficient way to create multi-task models by combining these fine-tuned models at the parameter level, without the need for training data or joint training on multiple datasets. Existing merging approaches typically involve scaling the parameters model-wise or integrating parameter importance parameter-wise. Both approaches exhibit their own weaknesses, leading to a notable performance gap compared to multi-task fine-tuning. In this paper, we unify these seemingly distinct strategies into a more general merging framework, and introduce Dynamic Fisher-weighted Merging (DF-Merge). Specifically, candidate models are associated with a set of coefficients that linearly scale their fine-tuned parameters. Bayesian optimization is applied to dynamically adjust these coefficients, aiming to maximize overall performance on validation sets. Each iteration of this process integrates parameter importance based on the Fisher information conditioned by the coefficients. Experimental results show that DF-Merge outperforms strong baselines across models of different sizes and a variety of tasks. Our analysis shows that the effectiveness of DF-Merge arises from the unified view of merging and that near-optimal performance is achievable in a few iterations, even with minimal validation data.
Introduction
The current AI landscape is characterized by a proliferation of "expert" models—large language models (LLMs) fine-tuned for specialized domains such as coding, legal analysis, or medical diagnosis. While Multi-Task Learning (MTL) is the gold standard for creating a single model capable of performing across multiple domains, it is often computationally prohibitive and requires simultaneous access to all training datasets, which may be proprietary.
Model merging has emerged as a cost-effective alternative, attempting to combine the weights of several expert models into a single multi-tasker. However, simple weight averaging often suffers from parameter interference, where critical weights for one task are treated as noise by another. Existing methods generally fall into two categories: model-wise scaling (e.g., Task Arithmetic) or parameter-wise importance weighting (e.g., Fisher Merging). This paper introduces DF-Merge, a framework that unifies these strategies to mitigate interference and maximize multi-task performance.
Methodology
A Unified Framework for Merging
DF-Merge is built on the conceptual breakthrough that model-wise scaling and parameter-wise importance are not mutually exclusive but rather restricted versions of a more general framework. As illustrated in Figure 1, while Task Arithmetic focuses on scaling the influence of each expert and Fisher Merging focuses on protecting critical parameters, DF-Merge occupies the flexible middle ground by combining both approaches.
Bayesian Optimization and Dynamic Fisher Information
To find the optimal "blend" of models, DF-Merge treats the merging process as a black-box optimization problem. It employs Bayesian Optimization (BO) to search for the ideal scaling coefficients ($\lambda$) for each task.
The process is iterative, as shown in Figure 3. In each iteration, the framework:
- Scales the task-specific parameters based on current coefficients.
- Estimates Parameter Importance by calculating the Fisher Information at these specific scaled points. This "dynamic" estimation captures the local curvature of the loss landscape more accurately than static methods.
- Merges the models by weighting parameters according to their importance.
- Evaluates the resulting model on a small validation set to guide the next BO iteration.
Experimental Results
Setup and Datasets
The researchers evaluated DF-Merge using T5-base and T5-large architectures across six diverse NLP benchmarks, including PAWS (paraphrase identification), WikiQA (question answering), and Winogrande (coreference resolution). DF-Merge was compared against popular baselines such as TIES-Merging, DARE, and standard Task Arithmetic.
Key Findings
- Performance Gains: DF-Merge significantly outperformed the best baseline, achieving a +4.48 point improvement in average accuracy for T5-base and +1.73 points for T5-large. It effectively narrowed the gap with high-cost multi-task fine-tuning to approximately 3.5 points.
- Efficiency: Despite the iterative nature of BO, the method is remarkably fast, reaching near-optimal performance within 9 to 20 iterations.
- Data Sparsity: The framework is highly data-efficient; using as little as 5% of the validation data was sufficient to surpass traditional merging methods.
- Robustness: Unlike static methods that often fail on specific tasks (e.g., WikiQA), DF-Merge maintains consistent performance across all benchmarks by dynamically resolving parameter interference.
Conclusion
DF-Merge provides a robust and efficient framework for combining specialized AI models without the need for extensive retraining. By unifying scaling and importance-based merging through Bayesian Optimization, it achieves state-of-the-art results in model merging. Future work may explore moving beyond diagonal Fisher approximations to block-diagonal structures and investigating data-free merging by using expert models as pseudo-labelers.
Summary updated