Abstract
Tool learning enables Large Language Models (LLMs) to interact with the external environment by invoking tools, enriching the accuracy and capability scope of LLMs. However, previous works predominantly focus on improving the model's tool-utilizing accuracy and the ability to generalize to new, unseen tools, excessively forcing LLMs to adjust specific tool-invoking patterns without considering the harm to the model's general performance. This deviates from the actual applications and original intention of integrating tools to enhance the model. To tackle this problem, we dissect the capability trade-offs by examining the hidden representation changes and the gradient-based importance score of the model's components. Based on the analysis result, we propose a Component Importance-based Tool-utilizing ability Injection method (CITI). According to the gradient-based importance score of different components, it alleviates the capability conflicts caused by the fine-tuning process by applying distinct training strategies to different components. CITI applies Mixture-Of-LoRA (MOLoRA) for important components. Meanwhile, it fine-tunes the parameters of a few components deemed less important in the backbone of the LLM, while keeping other parameters frozen. CITI can effectively enhance the model's tool-utilizing capability without excessively compromising its general performance. Experimental results demonstrate that our approach achieves outstanding performance across a range of evaluation metrics.
Introduction: The Tool-Use Trade-off
To interact effectively with the real world, Large Language Models (LLMs) must be capable of invoking external tools such as calculators, search engines, or APIs. While standard fine-tuning on tool-learning datasets can teach these skills, it often results in a severe trade-off: a sharp decline in the model's general cognitive abilities, including mathematics, coding, and general knowledge retrieval. This phenomenon, known as catastrophic forgetting, suggests that current fine-tuning methods force the model into a rigid tool-calling mindset at the expense of its broader intelligence.
As shown in Figure 1, while full fine-tuning (FT) improves correctness in tool-utilizing tasks, it leads to a significant drop in accuracy across general benchmarks like GSM8K and HumanEval compared to the base Llama-3-8B-Instruct model.
Methodology: Diagnosing Capability Conflicts
The authors identified two primary drivers of this performance degradation:
- Co-directional Shift: Analysis of hidden representations revealed that after tool fine-tuning, models struggle to distinguish between tool-related and tool-unrelated inputs, applying tool-like transformations even to general tasks.
- Shared Component Importance: Using gradient-based importance scores, the researchers discovered that the linear layers most critical for tool use are largely the same layers critical for general tasks. Overwriting these "important" components destroys general knowledge, yet ignoring them prevents the model from learning complex tool logic.
Figure 3 illustrates the high correlation (often >0.8) between the importance of network components for tool use and other general capabilities like Math and Coding, confirming that these abilities "share the same real estate" within the model architecture.
The CITI Approach
To resolve these conflicts, the Component Importance-based Tool-utilizing ability Injection (CITI) method applies differentiated training strategies:
- Important Components (Top 20%): These are frozen to preserve general knowledge. Instead, Mixture-of-LoRA (MOLoRA) adapters are attached. A specialized router network classifies inputs as tool-related or unrelated, directing them to specific LoRA experts to prevent representation drift.
- Unimportant Components (Bottom 10%): These undergo standard full-parameter fine-tuning. Since these components store less general knowledge, updating them allows the model to learn the rigid syntax of tool calling without compromising core intelligence.
Experimental Results
CITI was evaluated using Meta-Llama-3-8B-Instruct, Phi-3-mini, and Mistral-7B-Instruct-v0.2 across several datasets, including API-Bank and ToolAlpaca for tool use, and GSM8K, HumanEval, and MT-Bench for general abilities.
- Preservation of General Abilities: On the API-Bank benchmark, CITI outperformed standard full fine-tuning (FT) by an average of 31.95% across general tasks while maintaining competitive tool-utilizing accuracy.
- Efficiency of Hybrid Training: The results highlight that standard LoRA often outperforms full FT in both domains, but CITI's targeted hybrid approach provides the best balance by routing complex logic through experts and syntax through the backbone.
Conclusion and Future Work
CITI demonstrates that tool-utilizing capabilities can be injected into LLMs without sacrificing general performance by respecting the internal importance hierarchy of the model's components. Future work will focus on optimizing the router mechanism in MOLoRA and developing more dynamic, execution-based evaluation metrics for tool use, as current string-matching metrics like ROUGE-L may not fully capture a model's functional tool-use proficiency.
Summary updated