Visual-Seeker: Towards Visual-Native Multimodal Agentic Search via Active Visual Reasoning

Zhengbo Zhang, Changtao Miao, Jinbo Su, Zhaowen Zhou, Chunxia Zhang, Xukai Wang, Ruiqi Liu, Kaiyuan Zheng, Jiansheng Cai, Bo Zhang, Zhe Li, Shiming Xiang, Ying YanarXiv 2026 · arXiv · OpenReview

TL;DR

This paper introduces Visual-Seeker, a novel multimodal agent designed for visual-native web search. Unlike previous agents that treat images as static inputs, Visual-Seeker actively reasons over visual details and dynamically gathers evidence, achieving state-of-the-art performance on challenging benchmarks. This advancement is crucial for agents to truly understand and navigate the visually rich web.

What Problem Does This Paper Address?

The paper tackles a significant limitation in current multimodal large language models (MLLMs) and deep search agents: their struggle with factual grounding in complex, open-world scenarios, particularly when visual information is key. While MLLMs are powerful, they often treat visual input as a passive add-on rather than an active component of reasoning. Existing multimodal search agents tend to rely on simple images with clear semantics and text-based evidence trails, limiting their ability to perform multi-hop, cross-modal reasoning. This leaves a gap in agents that can truly "see" and reason about the visually complex and dynamic web, which is filled with intricate, multi-entity images.

Methods

Visual-Seeker's core innovation lies in its "visual-native" approach to search, powered by active visual reasoning. Instead of just processing images, the agent actively attends to fine-grained visual details and dynamically harvests visual evidence throughout its search process.

The key components of their approach include:

  • Active Visual Reasoning Data Pipeline: To train Visual-Seeker, the authors developed a sophisticated pipeline to synthesize high-quality multimodal search trajectories. This pipeline involves:
  • Fine-grained Seed Entity Selection: Extracting relevant entities and their descriptions from complex, multi-entity images using MLLMs, with a three-step filtering and disambiguation process to ensure quality.
  • Multi-hop Trajectory Synthesis: Generating diverse and non-linear search paths by performing random walks on a Wikipedia knowledge graph, incorporating strategies like backtracking and cycle constraints to simulate realistic agent behavior.
  • Visual Evidence Injection: Crucially, this stage injects visual evidence into the synthetic VQA tasks. The agent is prompted to search for images and extract fine-grained visual details, forcing it to engage in pixel-level reasoning and integrate visual information into its search.
  • Visual-Seeker Agent: The agent is trained using supervised fine-tuning on the synthesized data. It operates within a ReAct-style loop, utilizing a set of five external tools: text_search, reverse_image_search, search_image, visit, and image_crop. The agent iteratively reasons, calls tools, and processes their results to answer visual questions.

Datasets

The paper introduces a newly synthesized dataset of 5,000 high-quality multimodal search trajectories. This dataset is crucial for training Visual-Seeker and is generated using their Active Visual Reasoning Data Pipeline. It includes:

  • VQA data without visual evidence injection.
  • VQA data with visual evidence injection.
  • Text-only QA trajectories.
  • VQA instances sampled from the FVQA dataset.

Key Insights and Ideas

  • Vision as an Active Participant: The most significant insight is the shift from treating vision as a static input to an active reasoning component. Visual-Seeker's ability to dynamically attend to and harvest visual evidence is key to its success.
  • The Power of Synthesized Data: The authors demonstrate that a carefully designed data synthesis pipeline can effectively train agents for complex visual reasoning tasks, overcoming the limitations of existing datasets. The "Visual Evidence Injection" stage is particularly effective in pushing the agent towards visual-native capabilities.
  • Multi-hop Reasoning on the Web: The agent's ability to perform multi-hop reasoning, not just on text but also by integrating visual cues, is essential for navigating the complexities of real-world web search.
  • Tool Orchestration for Visual Search: The success of Visual-Seeker highlights the importance of a well-defined set of tools and an agentic workflow that can intelligently orchestrate them for visual information retrieval and analysis.

Key Results

Visual-Seeker achieves state-of-the-art performance across five challenging multimodal agentic search benchmarks: MMSearch, MMSearch-Plus, BC-VL, MM-BC, and VisBrowse.

  • It outperforms existing open-source multimodal deep search agents.
  • It even competes with and surpasses several proprietary models, including GPT-5 and Gemini-2.5-Pro, on average.
  • On the MMSearch-Plus benchmark, which features complex multi-entity images, Visual-Seeker shows strong competitiveness, indicating improved visual understanding.
  • The ablation studies confirm the effectiveness of their data synthesis pipeline and the critical role of the image_crop and search_image tools for visual grounding and evidence collection.

Limitations and Future Work

  • Limitations:
    • The paper acknowledges that while their synthesized data is high-quality, it is still a simulation of real-world web interactions.
  • The performance on simpler benchmarks (like MMSearch) shows a lower number of tool interactions, suggesting potential for further optimization in tool usage.
  • Future Work:
    • The authors suggest that their approach can be extended to more complex scenarios requiring deeper visual understanding and reasoning.
  • Further exploration of how to more effectively integrate visual evidence into agentic workflows for even more robust and nuanced web search is a promising direction.

Summary updated

Abstract

Multimodal large language models (MLLMs) have demonstrated impressive capabilities in many visual tasks, but they often struggle with factual grounding when confronted with complex, open-world scenarios. While recent multimodal deep search agents attempt to address this issue by utilizing external tools, the visual-native search paradigm remains underexplored. Existing methods primarily rely on simple images with explicit semantics and text-only evidence trajectories, limiting the agent's ability to perform multi-hop, cross-modal reasoning and search. To address these limitations, we propose Visual-Seeker, a visual-native multimodal deep search agent via active visual reasoning. Rather than treating vision as a static input, our agent actively attends to fine-grained visual details, dynamically harvests visual evidence throughout the search process. To unlock its visual-native potential, we design an active visual reasoning data pipeline and synthesize 5K high-quality multimodal trajectories for model training. Extensive experiments demonstrate the state-of-the-art performance across five challenging multimodal search benchmarks, even surpassing several proprietary models, validating robust visual-native reasoning and search in real-world web environments. The code and data can be accessed at: https://github.com/ZhengboZhang/Visual-Seeker.
BibTeX
@misc{zhang2026visual,
  author = {Zhengbo Zhang and Changtao Miao and Jinbo Su and Zhaowen Zhou and Chunxia Zhang and Xukai Wang and Ruiqi Liu and Kaiyuan Zheng and Jiansheng Cai and Bo Zhang and Zhe Li and Shiming Xiang and Ying Yan},
  title = {{Visual-Seeker: Towards Visual-Native Multimodal Agentic Search via Active Visual Reasoning}},
  year = {2026},
  eprint = {2606.15231},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2606.15231},
}
Figure