MetaAgent: Toward Self-Evolving Agent via Tool Meta-Learning

Hongjin Qian, Zheng LiuarXiv 2025 · DOI · arXiv

Abstract

Modern Large Language Models (LLMs) often struggle with deep knowledge discovery tasks that require multi-step reasoning and dynamic tool integration. Traditional solutions rely on rigid, human-engineered workflows or expensive, data-intensive fine-tuning. This paper introduces MetaAgent, a self-evolving agent framework that learns to solve complex tasks through "meta tool learning." By starting with a minimal workflow and iteratively reflecting on past performance, MetaAgent builds a persistent local knowledge base and refines its reasoning strategies without any parameter updates. Experimental results on benchmarks like GAIA and WebWalkerQA demonstrate that MetaAgent matches or exceeds the performance of heavily engineered and fine-tuned agents.

Introduction: The Challenge of Deep Knowledge Discovery

While LLMs excel at simple information retrieval, they frequently falter when faced with tasks requiring long-horizon planning and the integration of disparate external tools. Current approaches to building autonomous agents generally fall into two categories: workflow-based agents (e.g., ReAct), which use rigid, manually designed instructions, and end-to-end trained agents, which require massive task-specific datasets for fine-tuning. Both approaches suffer from limitations in generalizability and high development costs.

MetaAgent addresses these issues by asking: Can an agent start with near-zero hardcoded instructions and naturally learn to use tools and plan tasks through practice alone?

Methodology: Meta Tool Learning

MetaAgent's architecture is built on the principle of self-evolution through three core components:

  1. Minimal Foundation: MetaAgent begins with a basic reasoning loop. When it encounters a bottleneck, it generates a natural language request for assistance. A separate "Tool Router" translates these requests into specific API calls (e.g., web search, code execution), allowing the central agent to focus entirely on high-level reasoning.
  2. Dynamic Context Engineering (Reflection): Upon task completion, the agent performs self-reflection and verified reflection (comparing results against ground truth). It distills these experiences into concise text-based rules that are dynamically injected into the prompt for future tasks.
  3. Self-Expansion (In-House Tools): Unlike standard agents that only see summarized tool outputs, MetaAgent stores all raw retrieved data in a persistent local vector database. Over time, this creates a rich "in-house tool" that the agent can query to cross-reference information from previous research sessions.
The MetaAgent workflow: transitioning from a novice with a minimal workflow to an expert through iterative reflection and tool construction.

Experimental Evaluation

The authors evaluated MetaAgent across three challenging benchmarks designed to test deep knowledge discovery:

  • GAIA (General AI Assistant): Tests multi-step reasoning and tool use (e.g., web browsing).
  • WebWalkerQA: Evaluates the ability to navigate complex subpages and synthesize multi-source answers.
  • BrowseComp: Focuses on extreme, long-horizon web navigation where answers are buried deep within search results.

Key Results and Insights

MetaAgent consistently outperformed expert-designed workflows and matched the performance of end-to-end trained models. On the GAIA benchmark, using the QwQ-32B model, MetaAgent achieved a score of 47.6%, significantly outperforming workflow baselines like Search-o1 (39.8%) and rivaling the RL-trained WebThinker (48.5%).

Performance comparison of MetaAgent against various baselines across GAIA, WebWalkerQA, and BrowseComp benchmarks.

Core Insights:

  • Learning Over Hardcoding: A simple "ask for help" loop combined with a memory of past mistakes allows an agent to organically develop superior workflows compared to rigid human-designed instructions.
  • Natural Language as an Interface: Using plain English for tool requests reduces the cognitive load on the LLM, allowing it to dedicate more context to reasoning rather than strict JSON formatting.
  • The Value of Raw Data: By maintaining a local database of raw tool outputs, the agent avoids the "iceberg problem" where critical information is lost in summarization, enabling deeper cross-referencing in later stages.

Limitations and Future Work

While MetaAgent shows remarkable self-evolution, it faces challenges regarding context window management as the bank of "lessons learned" grows. Additionally, the iterative reflection process introduces higher computational overhead per task compared to zero-shot agents. Future research may focus on methods for compressing or forgetting outdated experiences and "pre-warming" the agent's experience bank with annotated data to accelerate the learning curve in specific domains.

Summary updated

Abstract

In this work, we propose MetaAgent, an agentic paradigm inspired by the principle of learning-by-doing, where expertise is developed through hands-on practice and continual self-improvement. MetaAgent starts with a minimal workflow, equipped only with basic reasoning and adaptive help-seeking abilities. When a knowledge gap is encountered, MetaAgent generates natural language help requests, which are routed to the most suitable external tool by a dedicated tool router. As MetaAgent solves tasks, it continually conducts self-reflection and answer verification, distilling actionable experience into concise texts that are dynamically incorporated into future task contexts. Besides, MetaAgent autonomously builds in-house tools and a persistent knowledge base by organizing its tool-use history, further enhancing its ability to retrieve and integrate relevant information We term this continual, data-driven process as \textit{meta tool learning}, through which MetaAgent incrementally refines its reasoning and tool-use strategies, without changing model parameters or requiring further post-training. Evaluated on challenging knowledge discovery benchmarks, including GAIA, WebWalkerQA, and BrowseCamp, MetaAgent consistently outperforms workflow-based baselines and matches or exceeds end-to-end trained agents, demonstrating the promise of self-evolving agentic systems for robust, general-purpose knowledge discovery. We provide our source codes in https://github.com/qhjqhj00/MetaAgent.
BibTeX
@misc{qian2025metaagent,
  author = {Hongjin Qian and Zheng Liu},
  title = {{MetaAgent: Toward Self-Evolving Agent via Tool Meta-Learning}},
  year = {2025},
  eprint = {2508.00271},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2508.00271},
}
Figure