Language-model agents that call software tools (web APIs, databases, booking services) tend to act when they should not. They fill in a parameter the user never gave, or they call a tool that looks right but does the wrong thing. Today the fix is to ask the agent to double-check itself in its prompt. This proposal moves the check to the tool. Each tool is wrapped as an actor, meaning an isolated process that owns its own state and talks to the world only by receiving and sending messages. Before the tool runs, its message queue applies a guard. The guard checks two things. First, every argument must trace back to a concrete source, either something the user said or the output of another tool. Second, the arguments must satisfy the tool's own mined constraints, such as "parameter A requires parameter B" or "coordinates must be in WGS84." A message that fails the guard is never executed. Instead the tool replies with a structured deficit message that names exactly what is missing. That message travels back to whoever can supply the missing piece: an upstream tool, the planner, or the user.
The opportunity is concrete. On the Incomplete Conditions for Tool-use benchmark, a small model left to itself reaches only 20.90% F1 at recognizing a mismatched tool. It rises to 68.57% only when a five-step verification checklist is added to its prompt. Humans sit around 81% (Can Tool-augmented LLMs be Aware of Incomplete Conditions?). A prompt checklist is advisory, however, and it covers a single decision. It cannot stop a multi-step plan from executing a harmful call halfway through, and it cannot say which earlier step should have produced the missing value. Guards enforced at the tool can do both.
Core Claim
We make two claims. First, executable guards at the tool that check where each value came from match or beat prompt-level verification at abstaining correctly on single calls. They should do so while wrongly refusing fewer valid calls and while executing zero ungrounded calls. Second, and more distinctively, deficit messages that name the missing information let a multi-step tool chain repair itself. The tool holding the needed output supplies it without the whole plan being redone. This addresses a problem that earlier agent-based service composition explicitly left open.
The two directions play different roles. The actor model supplies the mechanism: isolated tools, a message queue as the enforcement point, and messages that carry their origin. Hewitt's argument in How To Use What You Know was that knowledge about how to use something should live inside the active entity rather than in a distant interpreter. The semantic interoperability gap supplies the problem, namely that API schemas state types but omit meaning, dependencies, and preconditions. It also supplies the idea of naming the gap precisely.
Relation to Prior Work
Three works motivate the problem. Making the Difference in Semantic Web Service Composition computes, with a logical subtraction operator, the "extra description" a consumer needs but a producer omits. It does this inside a centralized planner over a hand-built ontology of 105 concepts. Provider-Composer Negotiations for Semantic Robustness in Service Compositions decentralizes that repair into negotiating agents and reports search cost linear in the number of agents. Its authors concede that it treats each link in isolation and cannot handle a fix on one link that breaks the next. We borrow their central idea of naming the missing piece as a message. We replace their hand-written ontologies with constraints mined from ordinary API documentation, and we test the multi-link case they left open.
Can Tool-augmented LLMs be Aware of Incomplete Conditions? supplies the benchmark and the strongest competing method, its Structured Verification checklist. It is also limited to single-turn, prompt-level decisions.
LlamaRestTest supplies part of the method. Its fine-tuned 8B model extracts inter-parameter dependencies, meaning rules that tie one parameter's validity to another's, from documentation. It learns further constraints from server error messages. We reuse the extraction step to populate guards. We deliberately do not learn by failing against live servers, because in stateful systems such as the transactional runtime of Cloud Actor-Oriented Database Transactions in Orleans a trial call can corrupt state. Any constraint learning from errors happens in a sandbox replay only.
API-Bank and ToolLLM constrain the design. Both treat tools as passive endpoints driven by one central planner. ToolLLM's tree search already recovers from errors by backtracking, so our gain has to come from avoiding the bad call in the first place and from targeted repair, not from recovery as such. Analyzing the Roles of Descriptions and Actions in Open Systems is the conceptual reason to keep guards next to each tool: in an open system a tool's valid preconditions change, and only the tool can keep them current.
Research Program
We would build a thin actor runtime in which every API from the benchmarks becomes a tool actor. Every message in this runtime carries an origin tag for each argument value. The tag records either a span of the user's words or a field produced by a named tool.
Each tool actor's guard has three layers. The first is schema validation. The second is origin checking: every required argument must have a tag pointing to text or output that actually contains the value, verified by string match or a small entailment model. The third is a set of executable dependency rules extracted from the tool's documentation by an IPD-style extractor. For tool-mismatch cases, the tool actor also compares the intent stated in the request against its own documented effect, using a small model. We expect this last layer to be the weakest part of the design.
When a guard fails, the actor emits a deficit message listing the missing or violated items. The runtime routes it by a simple rule. If an upstream tool's documented output type covers the missing item, the message goes to that tool. Otherwise it goes to the planner, which may ask the user or abstain.
The planner itself is unchanged: the same model and prompts as the baselines. The comparison therefore isolates where the verification happens.
Baselines
The naive reference is the bare planner with no verification: zero-shot tool calling on the same model. On ICT the reported figure for Gemma-3-4B on ToolBench API Replacement is 20.90% F1 (Can Tool-augmented LLMs be Aware of Incomplete Conditions?).
The strongest existing method on the abstention question is Structured Verification from the same paper. It is a five-step prompt checklist covering goal, required data, tool fit, sufficiency, and the final decision. It reaches 68.57% F1 on that split with Gemma-3-4B, against a human reference of about 81%. We would rerun it ourselves with identical planner models: Gemma-3-4B, which matches the reported setting, plus a Qwen2.5 model. It would get the same tool documentation our guards see and the same token budget per episode. We would first reproduce the 68.57% before comparing. Beating it means higher abstention F1 on both ICT perturbation types together with an equal or lower rate of refusing valid calls. A gain in F1 bought purely by refusing more often does not count.
For multi-step tasks, the reference points are API-Bank Level 3 results: 70% for GPT-4 and 22% for GPT-3.5-turbo (API-Bank). We also compare against ToolLLM's depth-first tree search with backtracking (ToolLLM), run with the same planner model and call budget.
Three ablations isolate the key ingredients. The first keeps schema validation only, which tests whether origin checking matters. The second removes deficit routing, so failed guards simply return an error to the planner. The third, and most important, moves the identical guards into a single central validator that checks every call before dispatch. This separates the value of the guards from the value of placing them inside tool actors.
Why not just…?
Why not just use Structured Verification with a stronger model? This is the most serious threat on single calls. For tool-mismatch cases, our intent-matching layer is itself a small model making the same judgment the checklist makes, so we may not win there, and we would say so. Where we expect a clear advantage is utterance removal. There the planner often invents a plausible value that looks fine to a self-check, but that value has no origin in the conversation, and the origin check refuses it deterministically. The per-perturbation split of ICT settles this directly.
Why not just validate against the JSON schema? An invented value for a removed parameter is well-typed, so schema validation passes it. The schema-only ablation measures how much of the gain comes from origin and dependency checks. If schema-only matches the full guard, the proposal collapses to a trivial result, and we would report that.
Why not just put the same guards in a central validator? For single calls this may be equivalent in accuracy, and we expect it largely will be. The actor placement must earn its keep on multi-step chains. There the tool holding the missing output can be asked directly instead of triggering a full replan, and guard knowledge stays with each tool as its documentation changes. If the central-validator ablation matches the actor version on multi-step success and call count, we would present the contribution as origin-checked guards plus deficit messages and drop the architectural claim.
Why not just let tree search backtrack? Backtracking recovers only after a call has executed. In a live database or booking system, that executed bad call is the failure. We therefore count executed ungrounded calls as a metric in their own right.
Proposed Experiments
The first experiment runs the full ICT benchmark (ddehun/ICT), covering 1,610 instances across both perturbation types on both the API-Bank and ToolBench portions. We report abstention F1, the rate of refusing valid calls on unperturbed originals, and the number of ungrounded calls executed.
The second experiment addresses the fact that no benchmark in our material tests incomplete conditions across several steps. We would build such a set by applying ICT's utterance-removal procedure to the 314 API-Bank dialogues, targeting parameters whose value should come from an intermediate tool output or from the user, and removing that value midway through the chain. The measures are end-to-end task success, the number of executed calls, planner tokens, and the fraction of deficits resolved by routing to an upstream tool versus the user.
The third experiment checks guard quality. We score the extracted dependency rules against the 551-parameter IPD dataset used by LlamaRestTest. We then measure how often false rules cause wrong refusals. For reference, LlamaRestTest reports 72.44% valid input generation versus 68.82% for GPT-3.5; it reports no standalone dependency-extraction accuracy that we could find on Lacuna.
Two checks are decisive. The architecture claim survives only if the actor version beats the central-validator ablation on multi-step success or call efficiency. The guard claim survives only if, on utterance removal, origin checking lifts abstention F1 above the reproduced Structured Verification number without raising the rate of wrongly refused valid calls.
References
- Can Tool-augmented Large Language Models be Aware of Incomplete Conditions?
- How To Use What You Know
- Making the Difference in Semantic Web Service Composition
- Provider-Composer Negotiations for Semantic Robustness in Service Compositions
- LlamaRestTest: Effective REST API Testing with Small Language Models
- Cloud Actor-Oriented Database Transactions in Orleans
- API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMs
- ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs
- Analyzing the Roles of Descriptions and Actions in Open Systems / 162