Abstract
Model merging offers a cost-effective alternative to full-scale retraining for combining specialized Large Language Models (LLMs). However, existing techniques often rely on manual hyperparameter tuning or computationally expensive evolutionary searches. This paper introduces Differentiable Adaptive Merging (DAM), a gradient-based optimization method that learns fine-grained merging coefficients. DAM achieves performance parity with evolutionary methods at a fraction of the computational cost while highlighting the continued efficacy of simple weight averaging as a baseline.
Introduction
Model merging aims to synthesize the specialized capabilities of multiple LLMs—such as mathematical reasoning and multilingual proficiency—into a single unified model. Traditionally, researchers have navigated two extremes: manual methods like "Model Soups" (simple averaging) or TIES-Merging, which require tedious hyperparameter tuning, and automated evolutionary algorithms, which are effective but require thousands of model evaluations.
DAM addresses this gap by treating merging as an optimization problem rather than a search problem, providing an automated strategy that is both precise and computationally efficient.
Methods: Differentiable Adaptive Merging
DAM introduces a trainable framework for model synthesis. Instead of applying a single global weight to a model, DAM utilizes column-wise scaling. Every column within every weight matrix of the source models is assigned a unique, trainable scaling coefficient ($c_{ij}$). This allows the optimization process to selectively boost or downweight specific features at a highly granular level.
To optimize these coefficients, the authors employ a multi-faceted objective function:
- KL Divergence: The primary driver, ensuring the merged model's output distribution remains close to the original specialized models.
- Cosine Similarity Loss: Encourages orthogonality between scaling factors to preserve the unique diversity of the source models.
- Regularization (L1/L2): Promotes stability and sparsity in the learned coefficients.
By using gradient descent, DAM identifies optimal merging recipes using only a small representative dataset, bypassing the massive overhead of evolutionary search.
Experimental Setup
The researchers evaluated DAM across two primary scenarios:
- Japanese & Mathematics: Merging
shisa-gamma-7b(Japanese) withWizardMathandAbel(Math) using the Ichikara Japanese instruction dataset and MetaMathQA. - Multilingual & Code: Merging Llama-3 variants for German, Korean, and SQL tasks using Fischerboot, Wikipedia-Korean, and WikiSQL datasets.
Performance was measured using standard benchmarks including the JP Language Model Evaluation Harness, MGSM, and SQL-Eval.
Key Results and Insights
DAM demonstrated significant improvements over traditional manual merging techniques and matched the performance of state-of-the-art automated methods.
Performance Parity with Lower Overhead
In the Japanese/Math case study, DAM achieved an average score of 0.62, matching Evolutionary Merging while significantly outperforming DARE-TIES (0.55). In multilingual tasks, DAM led in German (0.4434) and SQL coding (0.6125), proving its robustness across different domains.
The Resilience of "Model Soups"
A surprising finding was the effectiveness of simple linear averaging (Model Soups). When models share the same base and are sufficiently similar, simple averaging often achieves ~95% of the performance of complex automated methods. However, DAM proves superior when models are misaligned or initialized differently, as its adaptive scaling can navigate complex weight spaces that manual methods cannot.
Objective Function Dynamics
Ablation studies confirmed that while KL Divergence is the main performance driver, the addition of Cosine Similarity significantly improves the stability and robustness of the merge, reducing variance across different training runs.
Conclusion and Future Work
DAM represents a significant step toward automating the "black art" of model merging. By replacing expensive search with efficient gradient descent, it enables the creation of high-performance multi-task models with minimal compute. Future research will focus on expanding DAM beyond linear layers to more diverse architectures and exploring the limits of merging dozens of specialized "expert" models into a single cohesive system.
Summary updated