CITI: Enhancing Tool Utilizing Ability in Large Language Models without Sacrificing General Performance

Yupu Hao, Pengfei Cao, Zhuoran Jin, Huanxuan Liao, Yubo Chen, Kang Liu, Jun Zhao

Abstract: Tool learning enables Large Language Models (LLMs) to interact with the external environment by invoking tools, enriching their accuracy and capability scope. However, previous works predominantly focus on improving tool-utilizing accuracy and generalization to new tools, often forcing LLMs to adjust specific tool-invoking patterns without considering the harm to the model's general performance. This deviates from the original intention of integrating tools to enhance models. To tackle this problem, we dissect capability trade-offs by examining hidden representation changes and gradient-based importance scores of model components. Based on this analysis, we propose Component Importance-based Tool-utilizing ability Injection (CITI). CITI alleviates capability conflicts by applying distinct training strategies based on component importance: it utilizes Mixture-of-LoRA (MOLoRA) for critical components and fine-tunes less important backbone parameters while keeping others frozen. Experimental results demonstrate that CITI effectively enhances tool-utilizing capability without excessively compromising general performance.

The Challenge of Tool Learning

While LLMs are impressive, they are often limited by their training data, leading to hallucinations or an inability to interact with the real world. To address this, researchers fine-tune models to use external tools like search engines or calculators. However, standard fine-tuning often acts as a "bull in a china shop." When a model is forced to learn specific tool-invoking patterns, it frequently overwrites the parametric knowledge and reasoning capabilities it already possesses.

As shown in Figure 1, while full fine-tuning (FT) can improve tool-specific metrics, it often results in a catastrophic drop in general reasoning. For instance, a model might learn to call an API but suddenly lose its ability to solve middle-school math problems.

Comparison of tool-utilizing and general task performance between the base Llama-3 model and a fully fine-tuned (FT) version, highlighting the significant drop in general reasoning capabilities.

The CITI Framework

To solve this trade-off, the authors proposed CITI (Component Importance-based Tool-utilizing ability Injection), which consists of three main pillars:

  1. Importance Scoring: Using gradient-based analysis, the method ranks every linear component in the model. The researchers discovered that components important for tool use are often the same ones critical for general tasks.
  2. Mixture-of-LoRA (MOLoRA) for Critical Components: For high-importance components, the original weights are left untouched. Instead, multiple LoRA (Low-Rank Adaptation) experts are added. A specialized router network directs tool-related queries to these experts while keeping general queries on the "clean" original backbone path.
  3. Selective Full Fine-Tuning: For components deemed "unimportant" (those with minimal impact on general performance), the model allows full parameter fine-tuning to help it deeply absorb tool-calling formats.

Key Insights: Component Consistency and Representation Shifts

A pivotal discovery of this work is Component Consistency. The authors found that the parts of a model important for mathematics and coding are largely the same parts required for tool use. This explains why naive fine-tuning is so destructive: it attempts to rewrite the same "brain cells" for two different, potentially conflicting jobs.

Heatmap showing the correlation of component importance across different tasks, demonstrating that tool-utilizing abilities share critical components with general reasoning tasks like math and coding.

Additionally, the researchers identified a Co-directional shift. During standard fine-tuning, the model's internal representations for all tasks shift in the same direction, making it harder for the model to distinguish between tool-related and general inputs. CITI's routing mechanism effectively acts as a shield, preserving the model's original reasoning path for general tasks.

Experimental Results

CITI was evaluated across two primary domains: Tool Learning (using API-Bank and ToolAlpaca) and General Performance (using GSM8K, HumanEval, TriviaQA, and MT-Bench). CITI consistently outperformed standard LoRA and Full Fine-Tuning (FT) in preserving general intelligence:

  • General Task Preservation: On the API-Bank dataset, CITI was 31.95% better than full fine-tuning in average general performance.
  • Math and Logic Stability: While full fine-tuning caused Llama-3's GSM8K (math) score to crater from 79.45 to 26.38, CITI maintained a score of 77.33.
  • Tool Proficiency: Despite the focus on preservation, CITI remained highly competitive in tool use, often matching or exceeding the correctness scores of models that were fully fine-tuned specifically for tools.

Conclusion

CITI demonstrates that it is possible to "inject" specialized tool-using capabilities into LLMs without lobotomizing their general versatility. By identifying critical components and employing a routing-based architecture, CITI provides a robust framework for developing more capable AI agents that retain their foundational intelligence.

Summary updated

Abstract

Tool learning enables the Large Language Models (LLMs) to interact with the external environment by invoking tools, enriching the accuracy and capability scope of LLMs. However, previous works predominantly focus on improving model's tool-utilizing accuracy and the ability to generalize to new, unseen tools, excessively forcing LLMs to adjust specific tool-invoking pattern without considering the harm to model's general performance. This deviates from the actual applications and original intention of integrating tools to enhance model. To tackle this problem, we dissect the capability trade-offs by examining the hidden representation changes and the gradient-based importance score of model's components. Based on the analysis result, we propose a Component Importance-based Tool-utilizing ability Injection method (CITI). According to the gradient-based importance score of different components, it alleviates the capability conflicts caused by fine-tuning process by applying distinct training strategies to different components. CITI applies Mixture-Of-LoRA (MOLoRA) for important components. Meanwhile, it fine-tunes the parameters of few components deemed less important in the backbone of the LLM, while keeping other parameters frozen. CITI can effectively enhance the model's tool-utilizing capability without excessively compromising its general performance. Experimental results demonstrate that our approach achieves outstanding performance across a range of evaluation metrics.
BibTeX
@inproceedings{hao2025citi,
  author = {Yupu Hao and Pengfei Cao and Zhuoran Jin and Huanxuan Liao and Yubo Chen and Kang Liu and Jun Zhao},
  title = {{CITI: Enhancing Tool Utilizing Ability in Large Language Models without Sacrificing General Performance}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  url = {https://openreview.net/forum?id=P5vC3Br2mR},
}
Figure