Introduction
Supervised Fine-Tuning (SFT) is a critical step in tailoring Large Language Models (LLMs) for specialized domains such as law, medicine, and coding. However, practitioners often encounter the "jack-of-all-trades, master of none" dilemma. Specifically, three core challenges persist: Catastrophic Forgetting, where gains in one domain lead to losses in others; Low Efficiency, caused by reliance on heuristic data mixing rules (e.g., 1:1 ratios) that ignore the model's prior knowledge; and Rigid Domain Expansion, which lacks the flexibility to scale specialized expertise without degrading general reasoning.
To address these, we introduce VersaTune, a framework that probes a model's intrinsic knowledge distribution and dynamically adjusts data mixing ratios during training. VersaTune achieves nearly 30% improvement in multi-domain performance and enables specialized expansion while mitigating skill decay.
Methodology: The VersaTune Framework
VersaTune operates in two distinct phases designed to align training data with the model's internal state and optimize learning dynamics.
Phase 1: Domain Knowledge Detection
Before fine-tuning, the framework establishes a Knowledge Consistency baseline. By prompting the base model with a <BOS> (Beginning of Sequence) token, it generates thousands of sequences that are subsequently classified by an "expert" model (e.g., GPT-4). This process reveals the model's natural domain leanings, allowing the framework to align training data with pre-existing internal distributions—a strategy found to be more effective than uniform mixing.
Phase 2: Efficient Fine-Tuning
During SFT, VersaTune dynamically adjusts domain weights using two real-time metrics:
- Learnable Potential ($\gamma$): Quantifies the remaining "room for improvement" by comparing current loss to a theoretical mastery ceiling.
- Forgetting Degree ($\phi$): Monitors loss increases in non-target domains to detect and mitigate knowledge decay.
These metrics allow the framework to balance "Multi-Ability Fostering" (broad excellence) and "Flexible Domain Expansion" (targeted specialization).
Experimental Setup
The framework was evaluated using a diverse training set across six domains: Law (9.2k instances), Medicine (10.1k), Finance (68.9k), Science (8.8k), Code (20k), and General (49k). Performance was measured across a comprehensive suite of benchmarks, including LegalBench, MedMCQA, FinEval, SciEval, HumanEval, and AGIEval.
Key Results and Insights
VersaTune demonstrates significant advantages over static data composition strategies.
- Performance Gains: VersaTune achieved a 29.77% improvement in overall multi-domain performance compared to standard uniform mixing.
- Competitive Edge: A Qwen-2.5-32B model fine-tuned with VersaTune outperformed frontier models like GPT-4o and Claude 3.5 Sonnet in medical and multi-ability benchmarks by up to 4.76%.
- Mitigating Forgetting: In domain expansion scenarios, VersaTune reduced performance decay in non-target areas by 38.77% compared to traditional methods.
Core Principles
The success of VersaTune stems from two key insights. First, Knowledge Consistency proves that training is most efficient when data distributions match the model's pre-existing knowledge. Second, Dynamic Weighting treats training as an optimization problem, ensuring compute is not wasted on domains the model has already mastered. Furthermore, the authors noted Mutual Reinforcement between domains, such as coding improvements aiding logical structure in other fields.
Conclusion and Future Work
VersaTune provides a robust solution for training multi-capability LLMs efficiently. While currently reliant on proprietary models for initial domain detection and proxy models for mastery ceilings, future work aims to transition toward a fully self-supervised detection process. This framework represents a significant step toward creating specialized AI experts that retain their general-purpose utility.
Summary updated