LMTuner: An user-friendly and highly-integrable Training Framework for fine-tuning Large Language Models

Yixuan Weng, Zhiqi Wang, Huanxuan Liao, Shizhu He, Shengping Liu, Kang Liu, Jun ZhaoarXiv 2023 · DOI · arXiv

Abstract

Adapting Large Language Models (LLMs) to specific domains remains a significant engineering challenge, requiring expertise in distributed training and various optimization libraries. LMTuner is a modular, user-friendly framework that simplifies this pipeline into a single command: Let_Tune(). By integrating a GPT-4-powered interaction module with hardware-aware configuration logic, LMTuner enables researchers to train models ranging from 300M to 130B parameters with minimal coding effort.

Introduction

While LLMs have revolutionized natural language processing, their application in specialized fields like medicine or law often requires fine-tuning on domain-specific data. Currently, this process is fragmented, requiring researchers to manually coordinate tools like MegatronLM for parallelism, bitsandbytes for quantization, and PEFT for efficient tuning. This "coding burden" creates a high barrier to entry for non-experts. LMTuner addresses this by providing a highly integrable, automated framework that bridges the gap between raw hardware and a ready-to-train model.

System Architecture and Methods

LMTuner is built upon a three-pillar modular architecture designed for ease of use and high portability.

1. Interaction Module

This module acts as the system's "brain," utilizing GPT-4 (or ChatGPT) as a training assistant. It automatically detects the server's hardware environment—specifically GPU count and VRAM—using pynvml. It then engages the user in a natural language dialogue to determine the optimal training strategy, suggesting datasets and hyperparameters based on the available resources.

2. Training Module

The core training engine supports a wide array of state-of-the-art techniques, including Parameter-Efficient Fine-Tuning (PEFT) and Memory-Efficient Fine-Tuning (MEFT). Key supported methods include Low-Rank Adaptation (LoRA), Quantized LoRA (QLoRA), and LOw-Memory Optimization (LOMO). To handle long context windows, it incorporates Position Interpolation techniques like Xpos and NTK-Aware Scaled RoPE. By default, the system utilizes the memory-efficient Lion optimizer and DeepSpeed’s ZeRO technology.

The three-step workflow of LMTuner: from natural language interaction to automated training and inference.

3. Inference Module

Following training, this module facilitates deployment by supporting INT8 and INT4 quantization. This reduces latency and memory usage, making the fine-tuned models viable for production hardware.

Hardware-Aware Configuration and Insights

A core innovation of LMTuner is its ability to use high-reasoning LLMs to manage the training of other models. This compensates for a user's potential lack of deep learning engineering expertise. The system validates user requests against detected hardware; for instance, if a user attempts to full-parameter fine-tune a large model on limited VRAM, the Interaction Module will steer them toward memory-efficient alternatives like QLoRA.

Mapping diverse user requirements to specific datasets, models, and parameter-efficient fine-tuning (PEFT) strategies.

Datasets and Portability

LMTuner includes curated, ready-to-use datasets across several domains:

  • English: The LIMA dataset (1,030 high-quality samples).
  • Chinese: A translated version of LIMA supplemented with samples covering history and philosophy.
  • Medical: Approximately 60,000 consultation dialogues from CMCQA/MedDialog.
  • Custom Data: Support for local JSONL files.

For users configuring models on local machines for remote cluster training, LMTuner automatically generates a Readme.md containing all necessary shell commands and environment setups, ensuring high portability.

Experimental Results

The efficacy of LMTuner was demonstrated by training a medical LLM on the MedDialog dataset using two A6000 GPUs. The resulting Llama-7B model achieved a BLEU-4 score of 19.02, significantly outperforming zero-shot GPT-3 (7.52) and even the larger GLM-130B with Chain-of-Thought prompting (15.61). Furthermore, the framework successfully managed models up to 130B parameters on a single server, allowing novices to commence training runs in under five minutes.

Conclusion and Future Work

LMTuner successfully democratizes LLM fine-tuning by automating complex configuration tasks through natural language interaction. While the current version relies on OpenAI API keys for the Interaction Module, future iterations aim to integrate more diverse architectures and further refine automated hyperparameter optimization logic to capture even more niche user requirements.

Summary updated

Abstract

With the burgeoning development in the realm of large language models (LLMs), the demand for efficient incremental training tailored to specific industries and domains continues to increase. Currently, the predominantly employed frameworks lack modular design, it often takes a lot of coding work to kickstart the training of LLM. To address this, we present "LMTuner", a highly usable, integrable, and scalable system for training LLMs expeditiously and with minimal user-input. LMTuner comprises three main modules - the Interaction, Training, and Inference Modules. We advocate that LMTuner's usability and integrality alleviate the complexities in training large language models. Remarkably, even a novice user could commence training large language models within five minutes. Furthermore, it integrates DeepSpeed frameworks and supports Efficient Fine-Tuning methodologies like Low Rank Adaptation (LoRA), Quantized LoRA (QLoRA), etc., enabling the training of language models scaling from 300M to a whopping 130B parameters using a single server. The LMTuner's homepage (https://wengsyx.github.io/LMTuner/)and screencast video (https://youtu.be/nsXmWOmN3rE) are now publicly available.
BibTeX
@misc{weng2023lmtuner,
  author = {Yixuan Weng and Zhiqi Wang and Huanxuan Liao and Shizhu He and Shengping Liu and Kang Liu and Jun Zhao},
  title = {{LMTuner: An user-friendly and highly-integrable Training Framework for fine-tuning Large Language Models}},
  year = {2023},
  eprint = {2308.10252},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2308.10252},
}
Figure