TL;DR
LLM agents exhibit significant, systematic bias when choosing between functionally identical tools, often favoring specific providers or tools listed earlier in the prompt. This paper introduces a new benchmark to quantify this bias, uncovers its root causes (such as semantic alignment and pre-training exposure), and proposes a lightweight filtering-and-sampling mitigation strategy that restores marketplace fairness without sacrificing task performance.
What Problem Does This Paper Address?
As large language models (LLMs) transition into autonomous agents, they rely heavily on external APIs to interact with the world—whether fetching the weather, translating text, or validating emails. In large tool marketplaces like RapidAPI, multiple providers often offer functionally equivalent tools. Ideally, an LLM agent should select among these identical choices neutrally or uniformly.
However, the authors discover that LLMs suffer from systematic tool-selection bias. Instead of choosing fairly, models frequently fixate on specific providers due to superficial metadata or prompt positioning. This introduces critical market unfairness for third-party developers (especially under pay-per-request pricing), inflates operational costs, and risks degrading user experience if a slower or less reliable tool is arbitrarily preferred.
Methods
The authors tackle the problem through a three-part framework:
- Bias Quantification: They define tool-selection bias using Total Variation (TV) distance, comparing a model's empirical selection rate against an ideal uniform distribution. This is split into API-level bias ($\delta_{API}$) and positional bias ($\delta_{pos}$).
- Explaining the Drivers: To find out why LLMs prefer certain tools, they conduct three types of controlled experiments:
- Attribute-Level Analysis: Evaluating features like description length, parameter count, and tool age using linear regression and random forests.
- Metadata Perturbations: Systematically scrambling or shuffling tool names, descriptions, and parameter schemas to isolate what the model responds to.
- Biased Continued Pre-Training (CPT): Intentionally saturating a training corpus (~3.5M tokens) with a specific tool's metadata to see if raw exposure frequency implants a preference.
- Mitigation Strategy: They propose a lightweight debiasing module. Instead of letting the main LLM choose directly from the full list, a smaller model (Qwen3-14B) filters the candidates down to a subset of valid tools. The system then samples uniformly at random from that subset, ensuring equal selection probability.
Datasets
The authors built their benchmark on top of the ToolLLM pipeline, which contains real-world APIs scraped from RapidAPI. They introduced a novel, curated benchmark consisting of 10 clusters of functionally interchangeable APIs (e.g., Email Validation, QR Code Generation, Geocoding). Each cluster contains 5 distinct APIs and 100 balanced, provider-agnostic user queries, totaling 1,000 evaluation pairs.
Key Insights and Ideas
- Semantic Alignment Rules: The strongest driver of tool selection is the surface-level semantic similarity between the user's query and the tool's description.
- Descriptions Matter More Than Names: Scrambling tool descriptions completely overhauls the model's choices, whereas changing literal tool names mostly just adds noise.
- Position Bias is a Fallback: When a model lacks a strong semantic preference for a specific API, it defaults heavily to positional bias, favoring whatever tool is listed first in the prompt context.
- Pre-training Satiation: Saturating pre-training data with a single endpoint significantly boosts its selection rate (yielding a 20× relative gain), proving that training data exposure directly seeds these biases.
Key Results
- Widespread Bias: All seven evaluated frontier and open-source LLMs (including GPT-4.1 mini, Claude 3.5 Sonnet, and Gemini 2.5 Flash) showed substantial bias. Average combined bias metrics ($\delta_{model}$) landed between 0.25 and 0.38, meaning that roughly 25% to 38% of the selection probability mass would need to be redistributed to achieve perfect fairness.
- Successful Mitigation: The proposed subset-selection strategy successfully flattened selection distributions. For GPT-4.1 mini, the combined bias dropped drastically from 0.380 to 0.094. Crucially, the filter maintained a micro-precision of ~0.996 and micro-recall of ~0.885, proving that fairness does not have to come at the cost of task accuracy.
Limitations and Future Work
The study is primarily limited by its scale: it relies on 100 synthetically generated queries per cluster, focuses only on English-language queries, and evaluates APIs drawn exclusively from RapidAPI. Furthermore, the attribute analysis left a significant portion of variance unexplained ($R^2 < 0.4$), indicating that more complex or hidden features are at play.
The authors suggest that future work should scale up the benchmark to more diverse clusters, explore the impact of preference tuning (like RLHF) on tool selection, and analyze the internal reasoning traces of models to better understand the mechanics behind these choices.
Summary updated