TL;DR
LLM agents typically operate with a fixed "configuration"—a static set of tools and strategies chosen before the task begins. ToolSelf breaks this rigidity by treating configuration as a dynamic, tool-updatable variable, allowing agents to mid-task swap their toolboxes, update sub-goals, and pivot strategies. By unifying execution and self-adaptation into a single policy, ToolSelf achieves a massive 28.8-point average improvement over static baselines across complex research and engineering benchmarks.
What Problem Does This Paper Address?
Building an LLM agent involves a "specialization-generalization" trade-off. If you give an agent a specialized toolbox and strict prompts for a specific task (specialization), it performs well but fails on anything else. If you give it every tool and a vague prompt (generalization), the agent gets overwhelmed by a massive action space and "diluted" guidance.
Existing solutions usually involve a separate "planner" model or a pre-execution optimizer. However, these methods decouple adaptation from execution. When a separate planner tells a worker what to do, fine-grained details about the environment are often lost in translation (information loss). Furthermore, if a task fails, it’s hard to tell if the fault lies with the plan or the execution (ambiguous credit assignment). The authors argue that for an agent to truly master long-horizon tasks, it must be able to "reconfigure" itself based on real-time feedback.
Methods
The core innovation is the ToolSelf paradigm, which introduces two main components:
- Tool-Driven Runtime Reconfiguration: The agent is equipped with a
reconfiguretool. When the agent realizes its current strategy is failing or its toolbox is insufficient, it invokes this tool. This triggers a loop where the agent summarizes its progress, requests specific changes (e.g., "I need a Python interpreter instead of a web searcher"), and generates a new "Stage Configuration" ($C_i$) to continue the task. - Configuration-Aware Two-stage Training (CAT): To teach the agent when and how to change its own settings, the authors use a two-step process:
- Stage I (Rejection Sampling Fine-tuning): The model is fine-tuned on successful trajectories generated by a larger "teacher" model (Qwen3-235B).
- Stage II (KTO Reinforcement Learning): The model undergoes RL using Kahneman-Tversky Optimization. Crucially, they use trajectory-level credit assignment: if the final task succeeds, every reconfiguration decision made along the way is rewarded. This internalizes self-adaptation as a native capability.
Datasets
The authors evaluated ToolSelf on five diverse, long-horizon benchmarks:
- FRAMES: A deep research dataset requiring multi-step Wikipedia retrieval and complex reasoning.
- xbench: Tests realistic deep search and multi-step retrieval.
- GAIA (and GAIA-WS): A general assistant benchmark covering science, geography, and history, often requiring multimodal tool use.
- SWE-bench Lite: A software engineering benchmark where agents must resolve real-world GitHub issues.
- MathVista: Used specifically as a training supplement to bolster multimodal reasoning.
Key Insights and Ideas
- Configuration as a State, Not a Constant: By moving the agent's "operating instructions" into the action space, the agent can proactively manage its own cognitive load and tool utility.
- Closing the Feedback Loop: ToolSelf elicits "emergent adaptivity." For example, in case studies, the agent was seen failing to parse a webpage, then autonomously "pivoting" its strategy to visit individual country pages—a behavior not explicitly hard-coded but learned through the reconfiguration tool.
- Context Management as a Choice: The agent can switch between "basic" context and "truncation" modes. This allows it to handle massive tool outputs (like long code files) without overflowing its memory, essentially "self-cleaning" its own history.
- Unified Policy Advantage: Keeping adaptation and execution in the same model (Integrated) outperformed delegating adaptation to a separate LLM (Decoupled) by up to 6.1 points, proving that the "execution-level" details are vital for making good planning decisions.
Key Results
- Massive Performance Gains: After CAT training, ToolSelf (using Qwen3-8B) improved from a 16.5% baseline to 45.3% average accuracy—a 28.8-point jump.
- Zero-Shot Strength: Even without specific training, the ToolSelf architecture outperformed specialized agents like
WebSailorandSWE-Agenton their respective home turfs. - Token Efficiency: ToolSelf is significantly more efficient than "patch-based" agents. On GAIA, it achieved higher accuracy while using roughly 4 to 20 times fewer tokens than competitors like
Co-Sight, thanks to its active context management. - Cross-Family Generalization: The method isn't just for Qwen; it showed similar performance boosts when applied to the MiniMax-M2.5 model family.
Limitations and Future Work
- Fixed Global Pool: While ToolSelf can choose which tools to use, it cannot yet create or synthesize entirely new tools for unseen tasks.
- Inference Latency: Each reconfiguration step requires additional model calls. In time-sensitive environments, this "thinking about thinking" might introduce unwanted delays.
- Future Directions: The authors suggest exploring dynamic tool synthesis (where the agent writes its own tools) and further optimizing the reconfiguration loop to reduce computational overhead.
Summary updated