WebWatcher: Breaking New Frontier of Vision-Language Deep Research Agent

Xinyu Geng, Peng Xia, Zhen Zhang, Xinyu Wang, Qiuchen Wang, Ruixue Ding, Chenxi Wang, Jialong Wu, Yida Zhao, Kuan Li, Yong Jiang, Pengjun Xie, Fei Huang, Jingren ZhouarXiv 2025 · DOI · arXiv

Introduction

The recent surge in "Deep Research" agents has showcased the potential for AI to autonomously navigate the web and synthesize information. However, these agents remain predominantly text-centric, often failing when tasks require interpreting visual data such as scientific diagrams, financial charts, or complex web layouts. To address this, researchers from Alibaba’s Tongyi Lab introduced WebWatcher, a multimodal deep research agent designed to bridge the gap between text-based search and visual reasoning.

The Challenge of Multimodal Research

Existing Vision-Language (VL) agents typically suffer from two primary limitations. First, many are "perception-only," capable of describing an image but unable to use that visual information as a pivot for multi-step investigations. Second, they often rely on rigid, template-based pipelines that lack the flexibility needed to navigate the "open-world" complexity of the live web. The core bottleneck identified by the authors is a lack of high-quality training data specifically designed for multi-step multimodal reasoning.

Methodology: Building a Visual Researcher

WebWatcher is built upon a three-pronged development strategy:

  1. QA-to-VQA Data Pipeline: To generate challenging training data, the team converted complex text-based QA pairs into Visual Question Answering (VQA) tasks. By "fuzzing" or masking key entities in questions (e.g., replacing a specific name with "the person in this image") and grounding them in authentic web images, the model is forced to use visual cues to initiate its research process.
  2. Cold-Start Supervised Fine-Tuning (SFT): The model was fine-tuned on 8,000 high-quality "ReAct" (Think-Act-Observe) trajectories. This teaches the agent the "grammar" of research—how to logically sequence tools such as OCR, web search, and code interpreters.
  3. Reinforcement Learning via GRPO: Using Group-Relative Policy Optimization (GRPO), the researchers refined the agent's decision-making. This stage optimizes tool orchestration, teaching the model when to continue searching and when it has gathered sufficient evidence.

WebWatcher operates with a comprehensive toolset, including Web Text and Image Search, a "Visit" tool for page summarization, a Code Interpreter for mathematical and data analysis, and an internal OCR engine.

Benchmarking and Results

The researchers introduced BrowseComp-VL, a new benchmark comprising 399 VQA pairs that require multi-step web navigation and cross-modal integration. Additionally, WebWatcher was evaluated against several high-difficulty benchmarks including Humanity’s Last Exam (HLE-VL), LiveVQA, and MMSearch.

Performance of WebWatcher across HLE-VL, BrowseComp-VL, LiveVQA, and MMSearch benchmarks compared to leading models.

As shown in the figure above, WebWatcher (specifically the 32B version based on Qwen2.5-VL) demonstrated state-of-the-art performance across all tested benchmarks. On the BrowseComp-VL benchmark, it achieved a score of 27.0%, doubling the performance of the GPT-4o baseline (13.4%). In the expert-level HLE-VL, it reached an average score of 18.2%, surpassing specialized reasoning models like o4-mini.

A notable finding is the "test-time scaling" effect. On the HLE benchmark, WebWatcher’s success rate increased from 13.6% (single attempt) to 41.9% when allowed 32 attempts (Pass@32). This suggests that systematic exploration and extended "thinking" time are crucial for solving highly complex problems.

Insights and Future Directions

The success of WebWatcher highlights the power of entity obfuscation in training data, which shifts the model's behavior from simple retrieval to active reasoning. Furthermore, Reinforcement Learning proved highly effective at teaching the agent the "cost" of tool use, leading to more precise and less redundant search behaviors.

Despite these gains, challenges remain. The multi-step reasoning process introduces higher inference latency compared to one-shot models. Additionally, agents can occasionally fall into reasoning loops or misinterpret tool outputs. Future work will focus on expanding the agent's capabilities to handle diverse file formats like complex PDFs and videos, while further optimizing the efficiency of the reasoning process.

Summary updated

Abstract

Web agents such as Deep Research have demonstrated superhuman cognitive abilities, capable of solving highly challenging information-seeking problems. However, most research remains primarily text-centric, overlooking visual information in the real world. This makes multimodal Deep Research highly challenging, as such agents require much stronger reasoning abilities in perception, logic, knowledge, and the use of more sophisticated tools compared to text-based agents. To address this limitation, we introduce WebWatcher, a multi-modal Agent for Deep Research equipped with enhanced visual-language reasoning capabilities. It leverages high-quality synthetic multimodal trajectories for efficient cold start training, utilizes various tools for deep reasoning, and further enhances generalization through reinforcement learning. To better evaluate the capabilities of multimodal agents, we propose BrowseComp-VL, a benchmark with BrowseComp-style that requires complex information retrieval involving both visual and textual information. Experimental results show that WebWatcher significantly outperforms proprietary baseline, RAG workflow and open-source agents in four challenging VQA benchmarks, which paves the way for solving complex multimodal information-seeking tasks.
BibTeX
@misc{geng2025webwatcher,
  author = {Xinyu Geng and Peng Xia and Zhen Zhang and Xinyu Wang and Qiuchen Wang and Ruixue Ding and Chenxi Wang and Jialong Wu and Yida Zhao and Kuan Li and Yong Jiang and Pengjun Xie and Fei Huang and Jingren Zhou},
  title = {{WebWatcher: Breaking New Frontier of Vision-Language Deep Research Agent}},
  year = {2025},
  eprint = {2508.05748},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2508.05748},
}
Figure