I-PTC: Interactive Programmatic Tool Calling for Stateful Tool-Augmented Agents

Huanzhi Mao, Chengkun Cao, Shuo Yuan, Joseph E. GonzalezarXiv 2026 · arXiv

TL;DR

I-PTC (Interactive Programmatic Tool Calling) transforms how LLM agents use tools by providing them with a persistent, notebook-like execution environment. By allowing agents to maintain state, inspect intermediate results, and "learn" from API errors, I-PTC significantly improves success rates on complex tasks while reducing token consumption by up to 3x compared to standard baselines.

What Problem Does This Paper Address?

Current Large Language Model (LLM) agents typically interact with tools in one of two ways. Conventional tool calling is a "chatty" process: the model issues one call, sees the result in its prompt, and then decides on the next call. This is inefficient for complex tasks involving loops or data aggregation.

Programmatic Tool Calling (PTC) improved this by letting models write entire scripts to handle workflows. However, standard PTC is usually "stateless"—every time the model runs a piece of code, it starts in a fresh sandbox. If a script fails halfway through due to a minor API quirk, the agent often has to restart from scratch. Furthermore, serializing every intermediate data point back into the prompt quickly exhausts context windows. The authors argue that real-world problem-solving is inherently stateful, and our tool-use interfaces should reflect that.

Methods

The authors introduce I-PTC, a framework that treats tool use as an interactive, stateful session rather than a series of isolated events. It consists of three core mechanisms:

  1. Stateful Code Execution: Much like a Jupyter Notebook, I-PTC maintains a persistent Python namespace. Variables, imports, and function definitions created in one "cell" remain available in the next. This allows agents to cache expensive API results and recover from errors without losing progress.
  2. On-Demand Tool Inspection: Instead of cramming hundreds of API definitions into the initial system prompt (which is expensive and distracting), agents can "pull" documentation only when needed using a syntax similar to IPython (e.g., api.method??).
  3. Self-Evolving APIs: When an agent encounters a recoverable error or discovers an undocumented API behavior, an "updating agent" generates a concise usage note. This correction is appended to the tool's documentation for future tasks, allowing the system to "learn" the environment's quirks over time.
Comparison of traditional tool calling, PTC, and I-PTC. I-PTC enables reusable variables and intermediate-result inspection.

Datasets

To test this framework, the authors introduced PTC-BENCH, a new benchmark featuring 240 production-style tasks across domains like calendar scheduling, travel planning, and organizational analysis. These tasks are designed to be "long-horizon," often requiring over 50 API calls to solve. The benchmark includes:

  • Main Queries: Complex single-turn orchestration.
  • Multi-turn Queries: Tasks requiring the reuse of information from a previous user follow-up.
  • Error-Recovery Queries: Tasks where deterministic API failures are injected to test the agent's resilience.

The authors also evaluated I-PTC on existing benchmarks like MCP-Universe and MCP-Bench to demonstrate generalization.

Key Insights and Ideas

  • State as a Token-Saver: One of the biggest "aha" moments is that keeping intermediate data in the execution environment's memory (as Python variables) is far more efficient than serializing it into the LLM's conversation context. This prevents "context pollution" and keeps the model focused on the high-level logic.
  • Exploratory Programming for Agents: By allowing agents to "probe" data (e.g., printing the first two items of a list) before committing to a massive processing script, I-PTC enables a more human-like, iterative debugging process.
  • API Documentation is Dynamic: The "Self-Evolving API" feature recognizes that human-written documentation is often incomplete. Treating the tool interface as a learnable state allows the agent collective to become more robust over time without retraining the underlying model.
The self-evolving API documentation process converts failure traces into concise addenda for future use.

Key Results

The results across ten different model families (including GPT, Gemini, and Claude) show a clear trend: I-PTC is both more accurate and more efficient.

  • Performance Boost: On PTC-BENCH, GPT-5.4's pass rate jumped from a dismal 18.6% with traditional tool calling to 80.0% with I-PTC.
  • Token Efficiency: I-PTC dramatically reduced "trajectory tokens" (the unique information the agent must process). For Claude Opus 4.7, trajectory tokens dropped from 30K to just 8K.
  • Resilience: In error-recovery scenarios, I-PTC models were significantly more likely to succeed because they could inspect the error, fix the specific line of code, and continue, rather than getting stuck in a loop of repeated failures.
Figure 1: Pass rate vs. trajectory tokens for ten models on PTC- BENCH

Limitations and Future Work

The authors acknowledge several hurdles for future research:

  • Stale State: Just like in human-used notebooks, persistent state can lead to "hidden state" errors where a variable from a previous cell causes unexpected behavior in a later one.
  • Model Capability Floor: Programmatic interfaces require strong reasoning. The authors found that models smaller than 4B parameters (like Qwen3.5-0.8B) actually perform worse with PTC than with simple tool calling, as they struggle to generate valid code.
  • Documentation Integrity: The self-evolving API mechanism is currently lightweight; if the "updating agent" writes an incorrect correction, it could potentially mislead future agents. Future work is needed to validate these updates before they are persisted.

Summary updated

Abstract

Large language models are increasingly capable of writing and executing code, suggesting a natural paradigm for tool-augmented agents: expressing tool use as programs rather than isolated calls. Programmatic tool calling (PTC) captures this idea through loops, batching, abstraction, and structured computation, but existing PTC approaches are largely stateless, limiting their ability to reuse intermediate results and adapt over time. This is problematic for real-world tasks, where tool environments and problem-solving processes are inherently stateful. We therefore present I-PTC, an interactive framework that executes stateful PTC: models issue code snippets that call tools, maintain reusable state, and evolve the tool interface through API corrections. To evaluate the effectiveness of I-PTC and the general PTC setting, we introduce PTC-Bench, a benchmark of long-tail, production-style user requests whose difficulty comes from large initial environments, many dependent API interactions, extended planning, and recovery across follow-ups and transient failures. Experiments on PTC-Bench show that I-PTC improves performance on complex multi-step tasks with large tool sets while substantially reducing token consumption over conventional tool-calling baselines and the original PTC baseline, highlighting stateful programmatic tool calling as a promising direction for scalable tool-augmented agents.
BibTeX
@misc{mao2026i,
  author = {Huanzhi Mao and Chengkun Cao and Shuo Yuan and Joseph E. Gonzalez},
  title = {{I-PTC: Interactive Programmatic Tool Calling for Stateful Tool-Augmented Agents}},
  year = {2026},
  eprint = {2611.01446},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.01446},
}
Figure