WebSailor-V2: Bridging the Chasm to Proprietary Agents via Synthetic Data and Scalable Reinforcement Learning

Kuan Li, Zhongwang Zhang, Huifeng Yin, Rui Ye, Yida Zhao, Liwen Zhang, Litu Ou, Dingchu Zhang, Xixi Wu, Jialong Wu, Xinyu Wang, Zile Qiao, Zhen Zhang, Yong Jiang, Pengjun Xie, Fei Huang, Jingren Zhou

Abstract

Autonomous AI agents represent a critical frontier in the pursuit of Artificial General Intelligence (AGI). However, a significant performance gap persists between open-source models and proprietary systems like OpenAI’s DeepResearch. This paper introduces WebSailor-V2, a 30B-parameter open-source model that narrows this chasm. By integrating a novel, graph-based synthetic dataset with a dual-environment reinforcement learning (RL) framework, WebSailor-V2 outperforms much larger models, such as the 671B DeepSeek-V3.1, and rivals top-tier closed-source agents on complex web research benchmarks.

Introduction

The development of autonomous web agents faces two primary hurdles: the lack of high-quality, non-monolithic training data and the difficulties of scaling reinforcement learning in real-world environments. Existing datasets often rely on simple tree-like logic that fails to capture the messy, interconnected, and uncertain nature of real-world research. Furthermore, training in live web environments is hindered by high latency, API costs, and non-stationary tool outputs. WebSailor-V2 addresses these challenges through a comprehensive post-training pipeline focused on data complexity and training stability.

Methodology

WebSailor-V2 utilizes the Reason + Act (ReAct) framework, enhanced by several key innovations in data generation and policy optimization.

SailorFog-QA-2 Synthetic Data

To move beyond simple "easy-to-hard" logic, the authors developed SailorFog-QA-2. This dataset is built upon a densely interconnected knowledge graph. Using random-walk sampling, the pipeline generates subgraphs with complex cyclic relationships and introduces intentional uncertainties. This forces the model to develop sophisticated reasoning and verification strategies rather than following linear, deterministic paths.

Dual-Environment Reinforcement Learning

To overcome the bottlenecks of real-world RL, a two-tier training environment was engineered:

  1. Simulated Environment: A high-fidelity, offline environment based on Wikipedia allows for rapid, low-cost algorithmic iteration.
  2. Real-World Managed Environment: A unified tool execution interface manages Query Per Second (QPS) constraints, caching, and error recovery, ensuring stable policy training despite the inherent volatility of the live web.

Tailored GRPO and Symbiotic Feedback

The model employs an adapted version of Group Relative Policy Optimization (GRPO). This implementation includes a token-level policy gradient loss and a "leave-one-out" strategy to minimize variance. Additionally, a symbiotic feedback loop dynamically filters synthetic data based on agent performance, allowing the data distribution and model policy to co-evolve.

Experimental Results

WebSailor-V2 was evaluated against a suite of "Deep Research" benchmarks. Despite its relatively modest 30B-parameter size, the model demonstrated remarkable efficiency and reasoning capabilities.

Comparative performance of WebSailor-V2 against open-source and proprietary models on BrowseComp-EN and xbench-DeepSearch benchmarks.

On the BrowseComp-EN benchmark, WebSailor-V2 achieved a score of 35.3, significantly outperforming DeepSeek-V3.1 (30.0) and Claude-4-Opus (18.8). Similar dominance was observed in xbench-DeepSearch, where it scored 73.7. In the highly challenging Humanity’s Last Exam (HLE), the model set a new high for open-source systems with a score of 30.6, surpassing OpenAI-o3 (20.2).

Discussion and Key Insights

The success of WebSailor-V2 highlights several critical insights for agent development:

  • Data Quality over Model Size: The 30B Mixture-of-Experts (MoE) model's ability to outperform models nearly 20 times its size suggests that the quality and distribution of training data are more impactful than raw parameter count.
  • The Role of Entropy: Unlike standard RL tasks where policy entropy typically drops during convergence, WebSailor-V2 maintained high entropy. This indicates that the agent retains a capacity for exploration, which is essential for navigating the non-stationary nature of the web where search results and page structures are constantly shifting.
  • Context Scaling: Success on complex tasks is highly dependent on context window size; the authors found that a 128k window was necessary for the long trajectories (up to 100 iterations) required in deep research.

Conclusion and Future Work

WebSailor-V2 demonstrates that a combination of sophisticated synthetic data and stabilized RL can produce open-source agents capable of competing with the world's most advanced proprietary systems. Future work will focus on expanding the toolset beyond the current four primary tools and improving the stylistic quality of the generated research reports to match the polish of proprietary competitors.

Summary updated

BibTeX
@inproceedings{li2026websailor,
  author = {Kuan Li and Zhongwang Zhang and Huifeng Yin and Rui Ye and Yida Zhao and Liwen Zhang and Litu Ou and Dingchu Zhang and Xixi Wu and Jialong Wu and Xinyu Wang and Zile Qiao and Zhen Zhang and Yong Jiang and Pengjun Xie and Fei Huang and Jingren Zhou},
  title = {{WebSailor-V2: Bridging the Chasm to Proprietary Agents via Synthetic Data and Scalable Reinforcement Learning}},
  booktitle = {International Conference on Learning Representations},
  year = {2026},
  url = {https://openreview.net/forum?id=HuP16O5SJf},
}
Figure