To Call or Not to Call: Diagnosing Intrinsic Over-Calling Bias in LLM Agents

Wei Shi, Ziheng Peng, Sihang Li, Xiting Wang, Xiang Wang, Mengnan Du, Na ZouarXiv 2026 · arXiv

TL;DR

Large Language Model (LLM) agents are notorious for "over-calling" external tools even when they shouldn't, such as when user requests lack crucial details. This paper diagnoses this behavior as an activation-independent "intrinsic bias" toward tool invocation and introduces a training-free, closed-form steering method called Adaptive Margin-Calibrated Steering (AMCS) to successfully mitigate it using Sparse Autoencoders.

What Problem Does This Paper Address?

As LLMs transition into autonomous agents, their ability to use external tools (APIs, search engines, databases) has become a core feature. However, an effective agent must know not only how to call a tool, but crucially, when not to.

Current LLM agents suffer from a severe "over-calling" bias: they eagerly invoke tools even when a request is underspecified. For example, if a user says "Play a song on Spotify" without providing the song name, the base model will often hallucinate a tool call instead of asking the user for clarification. On the When2Call benchmark, six popular models from three families show high accuracy when a tool call is actually required, but dismal accuracy on "no-call" scenarios, dragging overall performance down to the 55%–70% range.

Figure 1: Overview of over-calling, intrinsic bias, and AMCS. (a) Across six target models, call accuracy is high, but no-call accuracy is much lower, reducing overall accuracy; a representative case shows the model calling a tool despite missing required information. (b) Intrinsic Bias Hypothesis: at activation parity (m = 0), the decision still favors CALL; the neutral boundary shifts to m⋆< 0. (c) AMCS converts the diagnosed bias β0 into a counter-bias shift δ, aiming to reduce false calls while preserving valid calls.

Methods

The authors investigate this bias through the lens of mechanistic interpretability using the following pipeline:

  1. Sparse Autoencoders (SAEs): The authors train TopK SAEs on the residual streams of middle-to-late transformer blocks across six target models (Qwen3.5, Gemma-3, and Ministral-3). This allows them to decompose dense internal activations into sparse, interpretable features.
  2. Feature Ranking & Margin Definition: They isolate specific SAE features that track the model's CALL and NO_CALL decisions. They define a signed activation margin ($m$) as the difference between the mean activations of these two feature groups.
  3. The Intrinsic Bias Hypothesis (IBH): Rather than over-calling being driven purely by fluctuating activation levels, the authors hypothesize that the model's decision mapping carries an activation-independent CALL offset ($\beta_0$). Even at activation parity ($m=0$), the model still favors calling.
  4. Adaptive Margin-Calibrated Steering (AMCS): To causally validate and correct this bias, they introduce AMCS—a closed-form counter-bias shift applied directly along the SAE decoder directions during inference. This shifts the decision boundary back to neutrality without requiring model retraining or iterative parameter tuning.

Datasets

  • OpenWebText2: Used for Stage 1 pre-training of the SAEs to establish a broad, generalized feature basis.
  • When2Call: A benchmark specifically designed to evaluate tool-gating decisions. The training split (~10M tokens) was used for Stage 2 SAE fine-tuning, while the evaluation split was used to discover gating features and test the AMCS intervention.

Key Insights and Ideas

  • The Bias is Structural, Not Contextual: The paper proves that over-calling isn't just caused by confusing prompts. Even when internal evidence for calling vs. not calling is perfectly balanced, the model's internal math is hardwired to favor a tool call. The neutral decision boundary is structurally shifted into negative territory.
  • False Calls are Geometrically Separable: Within the subset of examples where the model decides to emit a tool call, "false calls" consistently sit closer to the parity line than "true calls." This geometric separation provides a clean lever for targeted intervention.
  • A Mechanistic "Effort" Knob: By adjusting the counter-bias shift, AMCS essentially creates a controllable "calling effort" knob. Developers can use this axis to tune how eager or conservative an agent should be about invoking external APIs based on the deployment context.

Key Results

  • Validation of IBH: The Intrinsic Bias Hypothesis was confirmed across all six evaluated models. The raw neutral decision boundaries were universally negative, mathematically demonstrating the inherent pro-call bias.
  • Significant No-Call Accuracy Boost: Applying AMCS successfully cancelled the diagnosed offset, raising no-call accuracy by 4 to 17 percentage points across five of the six models.
  • Preserved Tool-Call Capabilities: Crucially, this massive boost in no-call accuracy came with a negligible drop in valid tool-call accuracy (typically under 5 points), leading to an overall accuracy improvement of up to 5 points.
Figure 3: Discriminative validation across six target models. We train 5-fold cross-validated logistic probes with increasing numbers of selected SAE features and report mean AUROC.

Limitations and Future Work

  • Inference-Time Patch: AMCS is an inference-time correction rather than a training-side fix; it mitigates the symptom but does not alter or explain the training-time origin of the bias.
  • Model Dependency: On Gemma-3 models, true and false calls showed poor geometric separation along the SAE axes, meaning the SAE feature basis offered less leverage for intervention.
  • Scope of Evaluation: The empirical study centers primarily on the single-turn decisions within the When2Call benchmark. Future work is needed to explore how this bias manifests and can be controlled in complex, multi-turn, long-horizon agent architectures.

Summary updated

Abstract

LLM agents exhibit a consistent tendency to over-call, invoking tools even in situations where none is needed. On the When2Call benchmark, six models from three families show high call accuracy but much lower no-call accuracy, leaving overall accuracy in the 55%-70% range. We trace this to an Intrinsic Bias Hypothesis (IBH): the call/no-call decision mapping carries an activation-independent call offset, so the model favors call even at activation parity. Using Sparse Autoencoders (SAEs), we recover behavior-aligned feature bases for the call/no_call decision, reduce them to a signed activation margin, and estimate the offset directly. Across all six models, the model is decision-neutral only when no_call activation outweighs call activation, consistent with IBH. We then causally test IBH with Adaptive Margin-Calibrated Steering (AMCS), a closed-form counter-bias shift along SAE decoder directions. Cancelling the diagnosed offset mitigates over-calling and improves overall accuracy with a negligible drop in call accuracy. Our work recasts over-calling from an empirical phenomenon into a mechanistic object amenable to causal correction. Code is available at https://github.com/SKURA502/agent-sae/.
BibTeX
@misc{shi2026call,
  author = {Wei Shi and Ziheng Peng and Sihang Li and Xiting Wang and Xiang Wang and Mengnan Du and Na Zou},
  title = {{To Call or Not to Call: Diagnosing Intrinsic Over-Calling Bias in LLM Agents}},
  year = {2026},
  eprint = {2605.18882},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2605.18882},
}
Figure