AutoSurfer -- Teaching Web Agents through Comprehensive Surfing, Learning, and Modeling

Fazle Elahi Faisal, Qianhui Wu, Baolin Peng, Jianfeng GaoarXiv 2026 · arXiv

TL;DR

AutoSurfer is a novel framework that automatically generates high-quality training data for web-automation agents by systematically exploring websites using a breadth-first strategy. By grounding task synthesis and trajectory refinement in actual navigation paths, it significantly reduces model hallucinations and produces highly diverse training datasets. When used to fine-tune open-source vision-language models, AutoSurfer achieves a state-of-the-art 24.23% task completion accuracy on the WebArena benchmark, outperforming previous synthetic data generation methods.

What Problem Does This Paper Address?

Building multimodal Large Language Model (LLM) agents capable of automating complex web tasks (like booking a flight or managing a code repository) is a major goal in AI. However, these agents frequently fail on complex or unfamiliar websites because they lack high-quality web interaction data for training.

Existing automated data generation methods fall short in two main ways:

  1. Incomplete Exploration: They rely on random-walk exploration or propose tasks solely from a website's homepage, missing deeper functionalities hidden behind multi-level menus or dropdowns.
  2. Hallucinated Tasks: Because they analyze web pages in isolation, they often synthesize ambiguous or impossible tasks, leading to broken and unreliable training trajectories.

As a result, current agents struggle to map user intent to the correct sequence of graphical user interface (GUI) actions.

Comparison of AutoSurfer's capabilities against other state-of-the-art trajectory generators.

Methods

AutoSurfer introduces a three-stage pipeline designed to mimic how a human explores and learns a new website: Comprehensive Exploration, Complex Task Synthesis, and Trajectory Refinement.

Figure 1: The full architecture of AutoSurfer. Given a web environment, (a) AutoSurfer comprehensively explores the website to discover complex tasks and their trajectories, (b) AutoSurfer refines the discovered tasks and trajectories to improve correctness and quality while reducing hallucinations, (c) AutoSurfer converts the refined tasks and trajectories into an SFT dataset, and (d) AutoSurfer uses the SFT dataset to train a wLLM.
  1. Systematic Breadth-First Exploration: Instead of wandering randomly, AutoSurfer maintains a queue of discovered pages and action traces. It recursively expands hidden UI elements like menus and dropdowns. Crucially, it differentiates between fixed elements (like search bars) and dynamic elements (like product lists), sampling only a few dynamic elements to avoid redundant exploration. It also propagates knowledge across pages to prevent duplicating tasks.
  2. Trajectory-Grounded Task Synthesis: Rather than inventing tasks from isolated page content, AutoSurfer uses the entire navigation path leading up to a page to synthesize both action-oriented and information-seeking tasks. This ensures the generated tasks are realistic and achievable.
  3. Trajectory-Guided Refinement: The initial exploration paths can be messy or sub-optimal. AutoSurfer feeds these raw trajectories as "hints" to a web agent, which replays the task to optimize the path, remove redundant actions, and clean up the task descriptions.

Finally, these refined trajectories are converted into a supervised fine-tuning (SFT) dataset in ShareGPT format to train website-specific LLMs (wLLMs).

Datasets

The authors evaluated AutoSurfer using the WebArena benchmark, a highly realistic, self-hostable web environment. To evaluate single-website agent performance efficiently, they focused on four distinct domains, selecting one task per template:

  • CMS (Content Management System): 57 tasks
  • Gitlab (Code Repository): 56 tasks
  • Reddit (Forum): 26 tasks
  • Shopping (E-commerce): 55 tasks

Key Insights and Ideas

  • Human-like Exploration Matters: Shifting from random walks to a systematic breadth-first search allows the system to uncover deep site functionalities that prior methods completely missed.
  • Context Cures Hallucination: Grounding task generation in a full exploration trajectory rather than a single page view ensures that the synthesized tasks match real user behavior and site capabilities.
  • The Exploration-Refinement Gap: Exploration data alone is insufficient for training because the agent operates in a reduced action space (e.g., it rarely needs to scroll when just mapping a page). Task-specific refinement is necessary to force the agent to use the full action space (like scrolling and inputting data), providing much cleaner supervision for next-action learning.
Illustration of how AutoSurfer synthesizes tasks based on full exploration trajectories versus prior methods.

Key Results

The authors fine-tuned Qwen2.5-VL-7B-Instruct using data generated by AutoSurfer and compared it against other state-of-the-art data generators:

  • Overall Accuracy: AutoSurfer achieved an overall task completion accuracy of 24.23% (when trained on combined website data) and 23.71% (when training website-specific models, or wLLMs). This significantly outperforms prior methods like SynthAgent (19.59%), OS-Genesis (13.40%), and Explorer (4.64%).
  • Domain Success: AutoSurfer showed massive improvements in specific domains, particularly on Reddit, where the website-specific model achieved 34.62% accuracy compared to SynthAgent's 15.38%.
  • Task Diversity: Using text embeddings to calculate pairwise cosine similarity, AutoSurfer achieved a task diversity score of 0.7774, outperforming both OS-Genesis and SynthAgent (0.7459), proving it generates a broader distribution of unique tasks.

Limitations and Future Work

While AutoSurfer marks a notable advancement, some limitations remain:

  • Single-Website Focus: The current implementation is primarily optimized for single-website tasks and excludes multi-domain workflows (such as tasks requiring Google Maps or cross-site navigation).
  • Baseline Agent Constraints: To isolate the impact of data quality, the authors used a relatively simple LLM-based agent for evaluation. Consequently, the absolute accuracies across all tested models remain low compared to commercial, heavily engineered agent frameworks.

For future work, the authors suggest using AutoSurfer’s refined trajectories as an initialization step for reinforcement learning (RL), combining offline synthetic data with online interaction rewards to tackle longer-horizon, multi-website tasks.

Summary updated

Abstract

Recent advances in multimodal large language models (LLMs) have revolutionized web agents that can automate complex tasks on websites. However, their accuracy remains limited by the scarcity of high-quality web trajectory training data. Existing automatic trajectory generation methods suffer from incomplete website coverage due to homepage-based task proposals or random-walk exploration. Such methods often result in hallucinated or ambiguous task synthesis that lead to incomplete and unreliable trajectory generation. Here, we present AutoSurfer, a comprehensive web trajectory generator that addresses these limitations through three key innovations. First, AutoSurfer employs a systematic breadth-first exploration strategy that maintains a queue of discovered pages and action traces, propagates knowledge across pages to avoid redundant exploration, and recursively expands multi-level graphical user interface elements - closely resembling how a human would learn a new website. Second, AutoSurfer leverages the exploration trajectory to guide task synthesis, reducing hallucinations by grounding complex tasks in actual navigation paths rather than isolated actions or page content alone. Third, AutoSurfer uses the same exploration trajectory as hints to steer a web agent toward more accurate and reliable trajectory refinement. Together, these innovations enable AutoSurfer to comprehensively cover a website's action space and generate data suitable for training website-specific LLMs. We evaluate AutoSurfer on the WebArena benchmark by fine-tuning Qwen2.5-VL-7B-Instruct and demonstrate that it outperforms state-of-the-art methods - Explorer, OS-Genesis, and SynthAgent - achieving up to 24.23% overall task completion accuracy compared to 19.59% for the best prior method. Further, task diversity analysis demonstrates that AutoSurfer yields a more diverse distribution of synthesized tasks.
BibTeX
@misc{faisal2026autosurfer,
  author = {Fazle Elahi Faisal and Qianhui Wu and Baolin Peng and Jianfeng Gao},
  title = {{AutoSurfer -- Teaching Web Agents through Comprehensive Surfing, Learning, and Modeling}},
  year = {2026},
  eprint = {2604.27253},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2604.27253},
}

Loading related content...

Figure