Do LLMs Know Tool Irrelevance? Demystifying Structural Alignment Bias in Tool Invocations

Yilong Liu, Xixun Lin, Pengfei Cao, Ge Zhang, Fang Fang, Yanan CaoarXiv 2026 · arXiv

TL;DR

Large Language Models (LLMs) frequently invoke completely irrelevant tools if the user's query attributes happen to neatly fit the tool's expected parameters. This paper identifies this flaw as "structural alignment bias," introduces a benchmark called SABEval to measure it, and proposes a novel attention-scaling intervention that cuts these errors nearly in half without hurting general performance.

What Problem Does This Paper Address?

When deploying LLMs with external tools (function calling), knowing when not to call a tool is just as critical as knowing when to call one. Previous benchmarks evaluated tool rejection by randomly pairing queries with irrelevant tools, which usually resulted in a structural mismatch (e.g., the tool expects a date, but the query provides a username).

However, the authors discover a hidden vulnerability: Structural Alignment Bias. If an irrelevant tool has parameters that perfectly match the attributes in a query, the LLM will blindly invoke it, ignoring the fact that the tool's actual function doesn't match the user's goal. For instance, asking about a game on the PlayStation Store might trigger a tool meant for the Nintendo Switch Store simply because both take a game_title and region.

Figure 1: Illustration of erroneous tool invocation driven by the model’s strong reliance on structural alignment.

Methods

To study and fix this issue, the authors develop three core techniques:

  1. SABEval Dataset Generation: Leveraging Object-Oriented Programming (OOP) principles of polymorphism to create "sibling tools"—tools that share identical parameter structures but perform mutually exclusive functions.
  2. Contrastive Attention Attribution (CAA): A new mechanistic interpretability method. Instead of relying on strict token-to-token alignment between counterfactual prompts, CAA aggregates attention weights at the "span level" (e.g., grouping tokens into 'user-query' or 'tool-definition'). It uses attribution patching (a first-order Taylor expansion) to trace how information flows through the model's attention heads during tool invocation decisions.
  3. Attention Scaling Intervention: Once CAA identifies the internal attention pathways responsible for "semantic checking" (refusal) versus "structural matching" (invocation), the authors dynamically scale the attention weights at inference time—amplifying the semantic pathway and suppressing the structural pathway.

Datasets

  • SABEval ($D_0$): The primary new benchmark containing 5,050 instances derived from 101 tool templates across 7 domains (finance, entertainment, health, etc.).
  • SABEval Extensions ($D_1 - D_4$): Four extended subsets where 1 to 4 additional parameters are injected to test how the degree of structural alignment affects model bias.
  • ACEBench: A standard tool-use benchmark used to verify that the proposed mitigation strategy doesn't degrade the model's baseline ability to use correct tools.

Key Insights and Ideas

  • Random pairing masks the problem: Standard evaluations report near-zero tool invocation errors on irrelevant tools because the structures don't match. SABEval reveals that when structures do match, error rates skyrocket.
  • The "Shortcut" Mechanism: LLMs treat structural alignment as a cognitive shortcut. As you add more matching parameters between the query and the tool, the model becomes exponentially more confident in making the wrong tool call.
  • Competing Internal Pathways: The model's internal decision-making is a tug-of-war between two distinct circuits: a semantic pathway (which focuses on query positions to verify relevance) and a structural pathway (which focuses heavily on output formatting instructions). Erroneous tool calls happen when the structural pathway overpowers the semantic one.
Figure 5: Semantic (left) and structural (right) pathways in Qwen3-8B from top-k attention weights (k = 23, corresponding to 2% of the attention heads).

Key Results

  • Error Scales with Alignment: On SABEval ($D_0$), tool invocation rates (TIR) jumped dramatically across all tested models (e.g., from 0.04% to 34.26% for Qwen3-8B). As the alignment degree increased ($D_4$), the error rate escalated up to 90.44%.
  • Causal Proof via Counterfactuals: When the authors manually broke the structural alignment by substituting or removing a single parameter, the tool invocation rate dropped sharply, proving that structure—not just textual similarity—drives the bias.
  • Effective Mitigation: The attention scaling strategy (CAA) achieved a 45.55% average reduction in erroneous tool calls across models like Qwen3 and ToolACE, vastly outperforming zero-shot prompting baselines (which only managed an 11.19% reduction). Crucially, general tool-use accuracy on ACEBench remained virtually unchanged.
Figure 7: TIR under different scaling coefficients for Qwen3-8B (k = 23).

Limitations and Future Work

  • Scope of Workflows: The study isolates single-turn tool invocations and does not yet explore how structural alignment bias propagates through multi-turn agentic workflows.
  • Origin of Bias: It remains unclear whether this bias is inherently born during the model's massive pre-training phase or if it is inadvertently introduced during tool-use fine-tuning (instruction tuning). Pinpointing the exact training stage responsible is left for future research.

Summary updated

Abstract

Large language models (LLMs) have demonstrated impressive capabilities in utilizing external tools. In practice, however, LLMs are often exposed to tools that are irrelevant to the user's query, in which case the desired behavior is to refrain from invocations. In this work, we identify a widespread yet overlooked mechanistic flaw in tool refusal, which we term structural alignment bias: Even when a tool fails to serve the user's goal, LLMs still tend to invoke it whenever query attributes can be validly assigned to tool parameters. To systematically study this bias, we introduce SABEval, a new dataset that decouples structural alignment from semantic relevance. Our analysis shows that structural alignment bias induces severe tool-invocation errors in LLMs, yet remains largely unaccounted for in existing evaluations. To investigate the internal mechanisms underlying this bias, we propose Contrastive Attention Attribution, which reveals two competing pathways for semantic checking and structural matching. The relative strength of these pathways drives LLMs' tool invocation decisions. Based on these findings, we further introduce a rebalancing strategy that effectively mitigates structural alignment bias, as demonstrated by extensive experiments, without degrading general tool-use capabilities.
BibTeX
@misc{liu2026do,
  author = {Yilong Liu and Xixun Lin and Pengfei Cao and Ge Zhang and Fang Fang and Yanan Cao},
  title = {{Do LLMs Know Tool Irrelevance? Demystifying Structural Alignment Bias in Tool Invocations}},
  year = {2026},
  eprint = {2604.11322},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2604.11322},
}
Figure