BiasBusters: Uncovering and Mitigating Tool Selection Bias in Large Language Models

Thierry Blankenstein, Jialin Yu, Zixuan Li, Vassilis Plachouras, Sunando Sengupta, Philip Torr, Yarin Gal, Alasdair Paren, Adel Bibi

TL;DR

LLM agents exhibit significant, systematic bias when choosing between functionally identical tools, often favoring specific providers or tools listed earlier in the prompt. This paper introduces a new benchmark to quantify this bias, uncovers its root causes (such as semantic alignment and pre-training exposure), and proposes a lightweight filtering-and-sampling mitigation strategy that restores marketplace fairness without sacrificing task performance.

What Problem Does This Paper Address?

As large language models (LLMs) transition into autonomous agents, they rely heavily on external APIs to interact with the world—whether fetching the weather, translating text, or validating emails. In large tool marketplaces like RapidAPI, multiple providers often offer functionally equivalent tools. Ideally, an LLM agent should select among these identical choices neutrally or uniformly.

However, the authors discover that LLMs suffer from systematic tool-selection bias. Instead of choosing fairly, models frequently fixate on specific providers due to superficial metadata or prompt positioning. This introduces critical market unfairness for third-party developers (especially under pay-per-request pricing), inflates operational costs, and risks degrading user experience if a slower or less reliable tool is arbitrarily preferred.

Figure 1: Tool-calling enables LLMs to act through external services, but the selection process introduces bias. Models may favor certain tools based on superficial metadata or position rather than relevance (here “weatherapi com” is preferred), leading to a potential degraded user experience and unfair concentration of calls. If such biases are systematic across frontier LLMs, they risk distorting entire tool marketplaces, disadvantaging functionally equivalent competitors.

Methods

The authors tackle the problem through a three-part framework:

  1. Bias Quantification: They define tool-selection bias using Total Variation (TV) distance, comparing a model's empirical selection rate against an ideal uniform distribution. This is split into API-level bias ($\delta_{API}$) and positional bias ($\delta_{pos}$).
  2. Explaining the Drivers: To find out why LLMs prefer certain tools, they conduct three types of controlled experiments:
  3. Attribute-Level Analysis: Evaluating features like description length, parameter count, and tool age using linear regression and random forests.
  4. Metadata Perturbations: Systematically scrambling or shuffling tool names, descriptions, and parameter schemas to isolate what the model responds to.
  5. Biased Continued Pre-Training (CPT): Intentionally saturating a training corpus (~3.5M tokens) with a specific tool's metadata to see if raw exposure frequency implants a preference.
  6. Mitigation Strategy: They propose a lightweight debiasing module. Instead of letting the main LLM choose directly from the full list, a smaller model (Qwen3-14B) filters the candidates down to a subset of valid tools. The system then samples uniformly at random from that subset, ensuring equal selection probability.

Datasets

The authors built their benchmark on top of the ToolLLM pipeline, which contains real-world APIs scraped from RapidAPI. They introduced a novel, curated benchmark consisting of 10 clusters of functionally interchangeable APIs (e.g., Email Validation, QR Code Generation, Geocoding). Each cluster contains 5 distinct APIs and 100 balanced, provider-agnostic user queries, totaling 1,000 evaluation pairs.

Key Insights and Ideas

  • Semantic Alignment Rules: The strongest driver of tool selection is the surface-level semantic similarity between the user's query and the tool's description.
  • Descriptions Matter More Than Names: Scrambling tool descriptions completely overhauls the model's choices, whereas changing literal tool names mostly just adds noise.
  • Position Bias is a Fallback: When a model lacks a strong semantic preference for a specific API, it defaults heavily to positional bias, favoring whatever tool is listed first in the prompt context.
  • Pre-training Satiation: Saturating pre-training data with a single endpoint significantly boosts its selection rate (yielding a 20× relative gain), proving that training data exposure directly seeds these biases.

Key Results

  • Widespread Bias: All seven evaluated frontier and open-source LLMs (including GPT-4.1 mini, Claude 3.5 Sonnet, and Gemini 2.5 Flash) showed substantial bias. Average combined bias metrics ($\delta_{model}$) landed between 0.25 and 0.38, meaning that roughly 25% to 38% of the selection probability mass would need to be redistributed to achieve perfect fairness.
  • Successful Mitigation: The proposed subset-selection strategy successfully flattened selection distributions. For GPT-4.1 mini, the combined bias dropped drastically from 0.380 to 0.094. Crucially, the filter maintained a micro-precision of ~0.996 and micro-recall of ~0.885, proving that fairness does not have to come at the cost of task accuracy.

Limitations and Future Work

The study is primarily limited by its scale: it relies on 100 synthetically generated queries per cluster, focuses only on English-language queries, and evaluates APIs drawn exclusively from RapidAPI. Furthermore, the attribute analysis left a significant portion of variance unexplained ($R^2 < 0.4$), indicating that more complex or hidden features are at play.

The authors suggest that future work should scale up the benchmark to more diverse clusters, explore the impact of preference tuning (like RLHF) on tool selection, and analyze the internal reasoning traces of models to better understand the mechanics behind these choices.

Summary updated

Abstract

Agents backed by large language models (LLMs) increasingly rely on external tools drawn from marketplaces where multiple providers offer functionally equivalent options. This raises a critical fairness concern: systematic bias in tool selection can degrade user experience and distort competition by privileging certain providers over others. We introduce a benchmark of diverse tool categories, each containing multiple functionally equivalent tools, to systematically evaluate tool-selection bias. Using this benchmark, we evaluate seven LLMs and show that substantial bias persists, with models either fixating on a single provider or disproportionately favoring tools that appear earlier in the context. To uncover the sources of this behavior, we conduct controlled experiments that isolate the effects of tool features, exposed metadata (name, description, and parameters), and pre-training exposure. We find that (1) semantic alignment between user queries and tool metadata is the strongest driver of selection; (2) small perturbations to tool descriptions can significantly shift choices; and (3) repeated pre-training exposure to a single endpoint amplifies provider-level bias. Finally, we propose a lightweight mitigation strategy that first filters tools to a relevant subset and then samples uniformly, substantially reducing selection bias while maintaining strong task coverage. Our results highlight tool-selection bias as a key obstacle to the fair deployment of tool-augmented LLM agents. Our code and benchmark are publicly available at https://github.com/thierry123454/tool-selection-bias.
BibTeX
@inproceedings{blankenstein2026biasbusters,
  author = {Thierry Blankenstein and Jialin Yu and Zixuan Li and Vassilis Plachouras and Sunando Sengupta and Philip Torr and Yarin Gal and Alasdair Paren and Adel Bibi},
  title = {{BiasBusters: Uncovering and Mitigating Tool Selection Bias in Large Language Models}},
  booktitle = {International Conference on Learning Representations},
  year = {2026},
  url = {https://openreview.net/forum?id=DEg4vvElYu},
}
Figure