ToolSelf: Unifying Task Execution and Self-Reconfiguration via Tool-Driven Emergent Adaptation

TL;DR

LLM agents typically operate with a fixed "configuration"—a static set of tools and strategies chosen before the task begins. ToolSelf breaks this rigidity by treating configuration as a dynamic, tool-updatable variable, allowing agents to mid-task swap their toolboxes, update sub-goals, and pivot strategies. By unifying execution and self-adaptation into a single policy, ToolSelf achieves a massive 28.8-point average improvement over static baselines across complex research and engineering benchmarks.

What Problem Does This Paper Address?

Building an LLM agent involves a "specialization-generalization" trade-off. If you give an agent a specialized toolbox and strict prompts for a specific task (specialization), it performs well but fails on anything else. If you give it every tool and a vague prompt (generalization), the agent gets overwhelmed by a massive action space and "diluted" guidance.

Existing solutions usually involve a separate "planner" model or a pre-execution optimizer. However, these methods decouple adaptation from execution. When a separate planner tells a worker what to do, fine-grained details about the environment are often lost in translation (information loss). Furthermore, if a task fails, it’s hard to tell if the fault lies with the plan or the execution (ambiguous credit assignment). The authors argue that for an agent to truly master long-horizon tasks, it must be able to "reconfigure" itself based on real-time feedback.

Methods

The core innovation is the ToolSelf paradigm, which introduces two main components:

  1. Tool-Driven Runtime Reconfiguration: The agent is equipped with a reconfigure tool. When the agent realizes its current strategy is failing or its toolbox is insufficient, it invokes this tool. This triggers a loop where the agent summarizes its progress, requests specific changes (e.g., "I need a Python interpreter instead of a web searcher"), and generates a new "Stage Configuration" ($C_i$) to continue the task.
  2. Configuration-Aware Two-stage Training (CAT): To teach the agent when and how to change its own settings, the authors use a two-step process:
  3. Stage I (Rejection Sampling Fine-tuning): The model is fine-tuned on successful trajectories generated by a larger "teacher" model (Qwen3-235B).
  4. Stage II (KTO Reinforcement Learning): The model undergoes RL using Kahneman-Tversky Optimization. Crucially, they use trajectory-level credit assignment: if the final task succeeds, every reconfiguration decision made along the way is rewarded. This internalizes self-adaptation as a native capability.

Datasets

The authors evaluated ToolSelf on five diverse, long-horizon benchmarks:

  • FRAMES: A deep research dataset requiring multi-step Wikipedia retrieval and complex reasoning.
  • xbench: Tests realistic deep search and multi-step retrieval.
  • GAIA (and GAIA-WS): A general assistant benchmark covering science, geography, and history, often requiring multimodal tool use.
  • SWE-bench Lite: A software engineering benchmark where agents must resolve real-world GitHub issues.
  • MathVista: Used specifically as a training supplement to bolster multimodal reasoning.

Key Insights and Ideas

  • Configuration as a State, Not a Constant: By moving the agent's "operating instructions" into the action space, the agent can proactively manage its own cognitive load and tool utility.
  • Closing the Feedback Loop: ToolSelf elicits "emergent adaptivity." For example, in case studies, the agent was seen failing to parse a webpage, then autonomously "pivoting" its strategy to visit individual country pages—a behavior not explicitly hard-coded but learned through the reconfiguration tool.
  • Context Management as a Choice: The agent can switch between "basic" context and "truncation" modes. This allows it to handle massive tool outputs (like long code files) without overflowing its memory, essentially "self-cleaning" its own history.
  • Unified Policy Advantage: Keeping adaptation and execution in the same model (Integrated) outperformed delegating adaptation to a separate LLM (Decoupled) by up to 6.1 points, proving that the "execution-level" details are vital for making good planning decisions.

Key Results

  • Massive Performance Gains: After CAT training, ToolSelf (using Qwen3-8B) improved from a 16.5% baseline to 45.3% average accuracy—a 28.8-point jump.
  • Zero-Shot Strength: Even without specific training, the ToolSelf architecture outperformed specialized agents like WebSailor and SWE-Agent on their respective home turfs.
  • Token Efficiency: ToolSelf is significantly more efficient than "patch-based" agents. On GAIA, it achieved higher accuracy while using roughly 4 to 20 times fewer tokens than competitors like Co-Sight, thanks to its active context management.
  • Cross-Family Generalization: The method isn't just for Qwen; it showed similar performance boosts when applied to the MiniMax-M2.5 model family.

Limitations and Future Work

  • Fixed Global Pool: While ToolSelf can choose which tools to use, it cannot yet create or synthesize entirely new tools for unseen tasks.
  • Inference Latency: Each reconfiguration step requires additional model calls. In time-sensitive environments, this "thinking about thinking" might introduce unwanted delays.
  • Future Directions: The authors suggest exploring dynamic tool synthesis (where the agent writes its own tools) and further optimizing the reconfiguration loop to reduce computational overhead.

Summary updated

Abstract

LLM-powered agentic systems excel at complex long-horizon tasks, but remain constrained by static configurations fixed before execution. Such rigidity forces a trade-off between domain-specific performance and cross-task generalization: strong priors and compact tool spaces aid specialization but weaken transfer, while task-agnostic workflows and broad action spaces expand coverage but dilute guidance. Existing pre-execution optimization, planner-worker orchestration, and configuration patching fall short of resolving this tension, as they decouple adaptation from execution, causing information loss, fragmented optimization, and ambiguous credit assignment. We propose ToolSelf, a **tool**-driven runtime **self**-reconfiguration paradigm that abstracts configuration updates as a standardized tool interface and unifies execution and adaptation within one policy's action space. The execution agent can dynamically update sub-goals, strategies, toolboxes, context, and context-management modes based on task progress and feedback. We further introduce **C**onfiguration-**A**ware Two-stage **T**raining (CAT), which combines rejection sampling fine-tuning with trajectory-level KTO reinforcement learning to internalize self-reconfiguration. Across diverse benchmarks, zero-shot ToolSelf rivals task-specialized agents; after CAT training, ToolSelf gains 28.8 points over the static-configuration baseline on average, illuminating a path toward emergent adaptivity that obviates manually injected guidance. The code is available at https://github.com/lian-tian-mo-zun/ToolSelf.
BibTeX
@inproceedings{2026toolself,
  title = {{ToolSelf: Unifying Task Execution and Self-Reconfiguration via Tool-Driven Emergent Adaptation}},
  booktitle = {RLxF @ ICML},
  year = {2026},
  url = {https://openreview.net/forum?id=BfIIJqEJvh},
}
Figure