Fine, I'll Merge It Myself: A Multi-Fidelity Framework for Automated Model Merging

Guinan Su, Jonas Geiping

Abstract

Reasoning capabilities represent a critical frontier for large language models (LLMs), but developing them requires extensive proprietary datasets and computational resources. One way to efficiently supplement capabilities is by model merging, which offers a promising alternative by combining multiple models without retraining. However, current merging approaches rely on manually-designed strategies for merging hyperparameters, limiting the exploration of potential model combinations and requiring significant human effort. We propose an Automated Model Merging Framework that enables fine-grained exploration of merging strategies while reducing costs through multi-fidelity approximations. We support both single and multi-objective optimization and introduce two novel search spaces: layerwise fusion (LFS) and depth-wise integration (DIS). Evaluating across a number of benchmarks, we find that the search autonomously finds 1) Merges that further boost single-objective performance, even on tasks the model has already been finetuned on, and 2) Merges that optimize multi-objective frontiers across tasks. Effective merges are found with limited compute, e.g. within less than 500 search steps.

Introduction

Model merging has emerged as a cost-effective strategy for enhancing Large Language Model (LLM) capabilities without the prohibitive expense of full-scale retraining. By blending the weights of specialized models—for instance, one optimized for mathematical reasoning and another for code generation—researchers can create versatile multi-task models. However, traditional merging techniques like Task Arithmetic or TIES-Merging typically rely on global hyperparameters applied uniformly across all layers. This "one-size-fits-all" approach is often suboptimal, as different layers contribute uniquely to specific tasks. Manually searching for optimal merge configurations is labor-intensive and frequently fails to identify the best possible performance trade-offs.

Methodology

To address these limitations, we introduce an Automated Model Merging Framework designed for efficient, fine-grained exploration of the merge space. The framework is built upon three core pillars:

  1. Multi-Fidelity Optimization: To mitigate the high cost of evaluating merge candidates on full benchmarks, we employ a "successive halving" strategy. Candidates are first screened on small data subsets (low fidelity), with only the most promising configurations proceeding to full-scale evaluation (high fidelity).
  2. Novel Search Spaces: We define two distinct search spaces to explore different merging paradigms:
  3. Layer-wise Fusion Space (LFS): Allows the optimizer to select different merging methods (e.g., SLERP, Task Arithmetic) and coefficients for specific groups of layers.
  4. Depth-wise Integration Space (DIS): Focuses on the inference pathway, enabling the framework to prune, repeat, or reorder layers from different source models to potentially create deeper or more efficient architectures.
  5. Multi-Objective Optimization: Utilizing the ParEGO algorithm, the framework identifies a "Pareto front" of models that balance conflicting objectives, such as simultaneous proficiency in coding and cross-lingual reasoning.
The proposed framework utilizes two novel search spaces: Layer-wise Fusion (LFS) for blending weights at the layer level, and Depth-wise Integration (DIS) for reconfiguring the model's architectural depth.

The framework was evaluated using a diverse set of benchmarks, including GSM8K and MATH for mathematical reasoning, MBPP and HumanEval for code generation, and MMLU for general knowledge.

Results and Discussion

Our experiments demonstrate that automated search can discover merge configurations that significantly outperform manually tuned baselines and even the original specialized source models.

Surpassing Specialized Models

A key finding is that the automated search identifies merges that exceed the performance of source models on the very tasks they were fine-tuned for. For example, merging a math-specialized model with a coding model resulted in a math performance boost beyond the original math-only model. As shown in Figure 2, the framework achieves superior accuracy across various merged model combinations and configurations.

Performance metrics across different merged model types (a) and configurations (b), highlighting the gains achieved through automated search compared to baseline models.

Key Performance Metrics

  • Benchmark Gains: The framework achieved a 4.24% improvement on the GSM8K math benchmark relative to the best individual source model.
  • Multi-Task Efficiency: In multi-objective scenarios, we observed a 6.86% average improvement across diverse task suites.
  • Search Efficiency: Effective merges were typically discovered within 500 search steps. In math tasks, only 17% of trials required full-budget evaluation, demonstrating the efficiency of the multi-fidelity approach.
  • Granularity and Ancestry: Ablation studies confirmed that increasing the granularity of layer groups improves results. Furthermore, the method is most effective when source models share a common architectural ancestor (e.g., Llama-2 variants), facilitating a compatible weight landscape.

Limitations and Future Work

While the framework offers a significant advancement in automated model development, several limitations remain. The current approach requires source models to share the same architecture and initialization. Merging heterogeneous architectures (e.g., Mistral and Llama) remains an open challenge. Additionally, merged models may inherit or amplify biases from their source components. Future research will focus on developing cheaper performance proxies and exploring cross-architecture merging techniques.

Summary updated

Abstract

Reasoning capabilities represent a critical frontier for large language models (LLMs), but developing them requires extensive proprietary datasets and computational resources. One way to efficiently supplement capabilities with is by model merging, which offers a promising alternative by combining multiple models without retraining. However, current merging approaches rely on manually-designed strategies for merging hyperparameters, limiting the exploration of potential model combinations and requiring significant human effort. We propose an Automated Model Merging Framework that enables fine-grained exploration of merging strategies while reducing costs through multi-fidelity approximations. We support both single and multi-objective optimization and introduce two novel search spaces: layerwise fusion (LFS) and depth-wise integration (DIS). Evaluating across a number of benchmarks, we find that the search autonomously finds 1) Merges that further boost single-objective performance, even on tasks the model has already been finetuned on, and 2) Merges that optimize multi-objective frontiers across tasks. Effective merges are found with limited compute, e.g. within less than 500 search steps.
BibTeX
@misc{su2025fine,
  author = {Guinan Su and Jonas Geiping},
  title = {{Fine, I'll Merge It Myself: A Multi-Fidelity Framework for Automated Model Merging}},
  year = {2025},
  eprint = {2502.04030},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2502.04030},
}
Figure