PORTS: Preference-Optimized Retrievers for Tool Selection with Large Language Models

Lorenzo Molfetta, Giacomo Frisoni, Nicolò Monaldini, Gianluca MoroEMNLP 2025

TL;DR

Large Language Models (LLMs) are increasingly being equipped with tools to enhance their capabilities, but efficiently selecting the right tool from a vast collection remains a challenge. This paper introduces PORTS, a novel method that fine-tunes tool retrievers by optimizing their alignment with LLM preferences, leading to significantly improved tool selection accuracy with minimal computational overhead.

What Problem Does This Paper Address?

LLMs are powerful, but they often struggle with complex tasks that require external knowledge or specific functionalities. Integrating LLMs with tools (like APIs) is a promising solution, allowing them to access real-world information or perform actions. However, as the number of available tools grows exponentially, LLMs face a "tool selection bottleneck." Simply retrieving tools based on keyword matching or general semantic similarity isn't enough, as these methods can be misaligned with how the LLM actually uses the tools. This misalignment can lead to incorrect tool choices, suboptimal performance, and increased computational costs. The paper addresses the challenge of effectively training tool retrievers to be more aligned with the downstream LLM's needs, especially in scenarios with many similar-looking but functionally distinct tools.

Methods

The core innovation of PORTS lies in its preference-optimized retrieval strategy. Instead of just optimizing for general relevance, PORTS fine-tunes a retriever by incorporating feedback from a frozen LLM. Here's a breakdown of the key components:

  • Goal-Directed Retrieval (REPLUG-like): PORTS builds upon existing methods that use an LLM's perplexity (confidence in predicting a correct tool call) as a signal to guide the retriever. This helps align the retriever's output distribution with the LLM's utility signal.
  • Preference Optimization Loss (Lpo): This is the novel contribution. PORTS introduces a contrastive loss that explicitly encourages the retriever to favor "positive" tool docstrings (those that lead to correct tool usage by the LLM) over "negative" ones. This is achieved by comparing the retrieval probabilities of a correct tool against incorrect ones, effectively learning to distinguish between semantically similar but functionally different tools. This is inspired by preference learning techniques like DPO and ORPO used in LLM fine-tuning.
  • Asynchronous Update of Hard Negatives: To maintain computational efficiency and adapt to evolving embedding spaces during training, PORTS periodically updates its selection of "hard" negative examples (tools that are semantically similar but incorrect).

Datasets

The authors evaluated PORTS on six popular tool-augmented datasets:

  • ToolBench: A large-scale dataset with diverse REST APIs across various domains.
  • API-Bank: A dataset focusing on general tools with a large number of domains.
  • APIBench: Tools for Java and Python programming.
  • BFCL-v2: A mix of Python and non-Python tools.
  • ToolE: Tools inspired by OpenAI plugins, covering various scenarios.
  • Octopus-v2: Android APIs.

For training, they adapted some datasets by decomposing multi-tool instances into single-tool examples and handling conversational inputs by removing past turns. They also created in-domain and out-of-domain variants for ToolE and Octopus-v2 to test generalization.

Key Insights and Ideas

  • LLM Preferences are Crucial for Tool Retrieval: The paper strongly suggests that simply matching query semantics to tool descriptions isn't enough. The retriever needs to understand how the LLM will actually use the tool, and PORTS' preference-based approach directly addresses this.
  • Contrastive Learning for Fine-Grained Distinctions: The core idea of using contrastive loss to differentiate between subtly different tools is powerful. This is especially important in domains where many tools might seem relevant but only one is truly optimal for the LLM's task.
  • Tool Docstring Quality Matters: The effectiveness of PORTS, like many retrieval methods, is sensitive to the quality and specificity of tool documentation. Vague descriptions can limit the gains.
  • Generalization to Unseen Tools is Enhanced: PORTS shows remarkable improvements not only for tools seen during training but also for entirely new tools, indicating its ability to learn robust retrieval strategies.

Key Results

PORTS consistently outperformed baseline retrievers across all tested datasets and LLMs.

  • Significant Recall@K Improvements: For seen tools, PORTS boosted average Recall@{1,2,3} by up to 71.66 percentage points and average NDCG@{1,3,5} by 70.16 percentage points.
  • Impressive Out-of-Domain Performance: Even with unseen tools, improvements were substantial, reaching +61.24 in Recall and +59.79 in NDCG.
  • Low Computational Overhead: PORTS achieves these gains with minimal additional computational cost compared to the baseline REPLUG.
  • Robustness: The method demonstrated robustness across different random seeds and hyperparameters.

Limitations and Future Work

The authors acknowledge two main limitations:

  • Sensitivity to Docstring Quality: PORTS' effectiveness is diminished in domains with vague or underspecified tool documentation.
  • LLM Query Overhead: While PORTS is memory-efficient, it requires repeated querying of a frozen LLM during training to generate guidance signals, which can introduce computational overhead in terms of time.

Future work could explore reducing reliance on LLM inference or developing more efficient approximations for guidance signals. The authors also suggest extending PORTS to other domains like biomedical discovery workflows and exploring metrics beyond recall for code generation scenarios.

Summary updated

BibTeX
@inproceedings{molfetta2025ports,
  author = {Lorenzo Molfetta and Giacomo Frisoni and Nicolò Monaldini and Gianluca Moro},
  title = {{PORTS: Preference-Optimized Retrievers for Tool Selection with Large Language Models}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2025},
}
Figure