TL;DR
The paper introduces MergeKit, an open-source toolkit designed to simplify and standardize the process of merging large language models (LLMs). By providing an efficient and extensible framework, MergeKit aims to democratize the creation of powerful, multitask LLMs from existing open-source checkpoints, reducing the need for extensive retraining and mitigating catastrophic forgetting.
What Problem Does This Paper Address?
The rapid proliferation of open-source LLMs has led to a diverse ecosystem of specialized models, each fine-tuned for specific tasks or domains. While this offers flexibility, managing and deploying numerous individual models is inefficient. Furthermore, fine-tuning separate models for each task can lead to "catastrophic forgetting," where a model loses its general capabilities, and requires substantial computational resources. Training models from scratch is also prohibitively expensive for many. Model merging offers a promising solution by combining the strengths of multiple pre-trained models into a single, more versatile model without requiring additional training. However, the process of merging models can be complex, requiring specialized knowledge and infrastructure. This paper addresses the need for a standardized, efficient, and accessible tool to facilitate this process.
Methods
MergeKit is presented as an open-source library that supports various model merging techniques. Its core design principles emphasize user-centricity, modularity, and scalability.
- YAML Configuration: Users define merge operations using YAML files, specifying merge methods, input models, and parameters. This makes the process repeatable, shareable, and accessible without requiring coding expertise.
- Modular Architecture: MergeKit employs a plug-and-play component system, allowing for easy integration of new merging methods. Key modules include
merge_methods/base.pyfor defining merge interfaces,graph.pyfor scheduling and execution, andarchitecture.pyfor handling model structures. - Scalability and Efficiency: The library utilizes an "out-of-core" approach, loading only necessary tensors into memory, enabling merging on resource-constrained hardware like personal laptops. It employs a Directed Acyclic Graph (DAG) to optimize the merging process, minimizing redundancy and resource usage.
- Novel Merging Methods: MergeKit introduces two novel methods:
- FrankenMerging (Passthrough): Allows piecewise combination of layers from multiple models to create models of unusual sizes.
- FrankenMoE (Mixture of Experts): Enables building MoE models from dense models using prompt-based routing or random gates.
- Existing Merging Techniques: MergeKit also supports established methods like Linear Interpolation (LERP), Spherical Linear Interpolation (SLERP), TIES merging, and DARE.
Datasets
The paper does not introduce new datasets. Instead, it focuses on merging existing pre-trained LLMs. The evaluation section mentions using medical benchmarks such as the US Medical License Exam (USMLE), Medical Multiple-Choice Question Answering (MedMCQA), and PubMed Question Answering (PubMedQA) to assess the performance of merged models.
Key Insights and Ideas
- Democratizing LLM Merging: MergeKit significantly lowers the barrier to entry for model merging, making advanced techniques accessible to a wider audience through its intuitive YAML interface and efficient backend.
- Efficiency is Key: The out-of-core processing and DAG-based scheduling are crucial for enabling large-scale model merging on commodity hardware, which is a significant bottleneck in the field.
- Extensibility for Future Innovation: By providing a well-defined architecture and clear interfaces, MergeKit encourages community contributions, fostering the development and integration of new merging strategies.
- Beyond Simple Averaging: The paper highlights that more sophisticated merging techniques, like SLERP and task arithmetic (TIES, DARE), can yield better results than simple weight averaging, especially when dealing with models fine-tuned from different initializations or on distinct tasks.
- Practical Impact: The success of merged models in specialized domains (e.g., medical) and general benchmarks demonstrates the power of this approach for creating versatile and high-performing LLMs.
Key Results
The paper presents a comparative analysis of merged models in the medical domain. For instance, merging Meditron-7B with Llama2-7B chat using LERP and SLERP techniques resulted in models that outperformed the base Meditron-7B model across various medical benchmarks (USMLE, MedMCQA, PubMedQA) and general benchmarks (Arc Challenge, HellaSwag, MMLU). Specifically, MeditronLlama-7B-Slerp achieved 39.20% on USMLE and 75.60% on PubMedQA, outperforming the base models. The paper also notes that MergeKit has facilitated the merging of thousands of models, contributing to some of the world's most powerful open-source checkpoints.
Limitations and Future Work
- Limitations:
- While MergeKit supports a wide range of merging techniques, the paper acknowledges that some advanced methods might still require custom implementation.
- The effectiveness of merging can still be sensitive to the choice of merging method and its hyperparameters, requiring experimentation.
- Future Work:
- The authors explicitly invite the community to contribute new merging strategies and enhancements to MergeKit.
- They are actively working to incorporate additional merging techniques like ZipIt, OT Fusion, and Git Rebasin.
- Further research into optimizing merging for models with different architectures and initializations is ongoing.
- The paper also touches upon the ethical considerations of LLM development, implying a need for continued focus on fairness, accountability, and transparency in merged models.
Summary updated