Gradientsys: A Multi-Agent LLM Scheduler with ReAct Orchestration

Xinyuan Song, Zeyu Wang, Siyi Wu, Tianyu Shi, Lynn AiarXiv 2025 · arXiv · OpenReview

Abstract

We present Gradientsys, a next-generation multi-agent scheduling framework that coordinates diverse specialized AI agents using a typed Model-Context Protocol (MCP) and a ReAct-based dynamic planning loop. At its core, Gradientsys employs an LLM-powered scheduler for intelligent one-to-many task dispatch, enabling parallel execution of heterogeneous agents such as PDF parsers, web search modules, GUI controllers, and web builders. The framework supports hybrid synchronous/asynchronous execution, respects agent capacity constraints, and incorporates a robust retry-and-replan mechanism to handle failures gracefully. To promote transparency and trust, Gradientsys includes an observability layer streaming real-time agent activity and intermediate reasoning via Server-Sent Events (SSE). We offer an architectural overview and evaluate Gradientsys against existing frameworks in terms of extensibility, scheduling topology, tool reusability, parallelism, and observability. Experiments on the GAIA general-assistant benchmark show that Gradientsys achieves higher task success rates with reduced latency and lower API costs compared to a MinionS-style baseline, demonstrating the strength of its LLM-driven multi-agent orchestration.

Introduction

Large Language Models (LLMs) have demonstrated remarkable reasoning capabilities, yet they often struggle when confronted with complex, multi-step real-world tasks. These tasks frequently require the use of specialized tools—such as parsing massive PDF documents, navigating dynamic web interfaces, or executing code—which fall outside the native capabilities of a standalone model.

Current multi-agent frameworks typically suffer from two primary limitations. First is the sequential bottleneck: systems like AutoGPT often operate in a single-agent loop, invoking one tool at a time. This approach is inherently slow and fails to exploit the inherent parallelism in many sub-tasks. Second is the issue of rigid workflows: many frameworks rely on static, hard-coded graphs or scripts, which lack the flexibility to adapt to tool failures or evolving task requirements.

To address these challenges, we introduce Gradientsys, a framework that treats multi-agent orchestration like an operating system. By managing agent capacity, handling failures, and executing tasks in parallel, Gradientsys provides a robust and scalable solution for complex AI-driven workflows.

System Architecture

The heart of Gradientsys is a centralized LLM Scheduler that utilizes the ReAct (Reasoning + Acting) paradigm. This scheduler maintains a "scratchpad" where it alternates between natural language reasoning and structured tool calls, allowing for dynamic planning and course correction.

Architecture Overview of Gradientsys

Key innovations within the Gradientsys architecture include:

  • Typed Model-Context Protocol (MCP): Acting as a "standardized interface," MCP allows tools to be integrated at runtime without custom glue code, ensuring high extensibility.
  • One-to-Many Parallel Dispatch: The scheduler identifies independent sub-tasks, triggering multiple agents concurrently through a threadpool-based dispatcher.
  • Streaming Execution: By intercepting partial token outputs, the system can begin executing tool calls before the LLM has even finished generating its full reasoning block, significantly reducing latency.
  • Observability Layer: Real-time "thoughts" and tool outputs are streamed to the user via Server-Sent Events (SSE), fostering transparency and trust.

Experimental Evaluation

We evaluated Gradientsys primarily on the GAIA (General AI Assistants) benchmark, which contains 466 challenging, real-world tasks requiring multi-hop reasoning. We also conducted a qualitative head-to-head comparison against Genspark using a 33-task subset of GAIA.

Key Results

Gradientsys was compared against a "MinionS-style" baseline, which represents a strong single-agent planner with one auxiliary worker.

Performance comparison on the GAIA benchmark

As shown in the results, Gradientsys achieved a 24.1% success rate, a significant improvement over the baseline's 15%. Furthermore, the average task latency was reduced from 52 seconds to 35 seconds, and the system operated at only 0.22x the cost of the baseline. In the head-to-head comparison with Genspark, Gradientsys was 4.3x faster and 3.3x cheaper while successfully completing more tasks (25 vs. 23).

Discussion and Insights

Our findings highlight several critical insights for multi-agent system design. First, the separation of orchestration and execution is vital for efficiency. By utilizing a high-level model (e.g., GPT-4) solely for scheduling and offloading heavy data processing to specialized tools or cheaper models, we achieved a 4.5x reduction in overall costs.

Second, we found that parallelism is essential for accuracy. Ablation studies indicated that sequential execution not only increases latency but also slightly decreases success rates, as longer sequential chains are more prone to reasoning "drift." Finally, the streaming planner approach minimizes idle time, providing a more responsive user experience.

Conclusion and Future Work

Gradientsys demonstrates that an LLM-driven scheduler, combined with standardized protocols and parallel execution, can significantly enhance the performance and cost-effectiveness of AI agents. Future work will focus on addressing knowledge gaps in specialized tools, refining the ReAct loop for extremely complex queries, and exploring "cascading" topologies where agents can autonomously spawn sub-agents. We also plan to implement cost-aware optimization to automatically select the most economical tool for any given sub-task.

Summary updated

Abstract

We present Gradientsys, a next-generation multi-agent scheduling framework that coordinates diverse specialized AI agents using a typed Model-Context Protocol (MCP) and a ReAct-based dynamic planning loop. At its core, Gradientsys employs an LLM-powered scheduler for intelligent one-to-many task dispatch, enabling parallel execution of heterogeneous agents such as PDF parsers, web search modules, GUI controllers, and web builders. The framework supports hybrid synchronous/asynchronous execution, respects agent capacity constraints, and incorporates a robust retry-and-replan mechanism to handle failures gracefully. To promote transparency and trust, Gradientsys includes an observability layer streaming real-time agent activity and intermediate reasoning via Server-Sent Events (SSE). We offer an architectural overview and evaluate Gradientsys against existing frameworks in terms of extensibility, scheduling topology, tool reusability, parallelism, and observability. Experiments on the GAIA general-assistant benchmark show that Gradientsys achieves higher task success rates with reduced latency and lower API costs compared to a MinionS-style baseline, demonstrating the strength of its LLM-driven multi-agent orchestration.
BibTeX
@misc{song2025gradientsys,
  author = {Xinyuan Song and Zeyu Wang and Siyi Wu and Tianyu Shi and Lynn Ai},
  title = {{Gradientsys: A Multi-Agent LLM Scheduler with ReAct Orchestration}},
  year = {2025},
  eprint = {2507.06520},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2507.06520},
}
Figure