Abstract
Language agents based on large language models (LLMs) have demonstrated great promise in automating web-based tasks. Recent work has shown that incorporating advanced planning algorithms, such as tree search, is advantageous over reactive planning. However, real-world web environments are rife with irreversible actions, undermining the feasibility of backtracking—a cornerstone of tree search. Overly relying on test-time search also hurts efficiency. We advocate for model-based planning for web agents, employing a world model to simulate and deliberate over the outcome of candidate actions before committing to them. We propose WebDreamer, a framework where LLMs serve as both world models and value functions. By training specialized LLMs (Dreamer-7B) with a scalable data synthesis pipeline, we demonstrate that WebDreamer achieves substantial performance improvements over reactive baselines. It is competitive with tree search in sandbox environments while being 4–5 times more efficient, and it effectively navigates real-world websites.
Introduction: The Unforgiving Web
Building autonomous agents to navigate the web is challenging because the internet is an "unforgiving" environment. Most advanced planning algorithms rely on backtracking—the ability to undo an action and try a different path. However, the real-world web is full of irreversible actions: one cannot easily "un-submit" a form, "un-buy" an item, or "un-delete" an account without significant consequences.
Current agents typically fall into two categories: reactive agents, which make short-sighted, one-step-at-a-time decisions, and tree search agents, which attempt real interactions to explore paths. The latter is slow, expensive, and often impossible on live websites. WebDreamer addresses this by using the internal knowledge of LLMs to simulate the web environment itself, allowing the agent to "dream" about consequences before acting.
The WebDreamer Framework
WebDreamer is a model-based planning framework that treats the LLM as both a "world model" and a "value function." The process is divided into two primary stages:
- Stage I: Simulation: Instead of acting directly on a website, the agent uses a simulation function to predict how a webpage will change after a specific action. For example, it might predict: "If I click 'Electronics', the page will show sub-categories like 'Computers'." A scoring function then evaluates if that predicted state brings the agent closer to the user's goal.
- Stage II: Execution: Once the best candidate action is identified through simulation, the agent executes it in the real environment.
Scalable Data Synthesis and Dreamer-7B
To enable smaller, more efficient models to act as world models, we developed a pipeline that "walks" the web randomly, capturing screenshots before and after actions. We utilized a massive vision-language model (Qwen2-VL-72B) to label these transitions with natural language descriptions.
Using this pipeline, we synthesized 3.1 million interaction instances across 1.2M unique URLs to train Dreamer-7B, a specialized 7B-parameter world model. This model is designed to be a compact expert at predicting web dynamics, tapping into the latent knowledge LLMs possess about web behavior from their pre-training.
Empirical Evaluation
We evaluated WebDreamer across several benchmarks:
- VisualWebArena (VWA): A sandbox environment with self-hosted shopping and social media sites.
- Online-Mind2Web & Mind2Web-Live: Benchmarks involving hundreds of tasks on real-world, live websites.
Key Results
- Performance Boost: WebDreamer (using GPT-4o) improved success rates by 34.1% on VisualWebArena and up to 42.3% on real-world online benchmarks compared to reactive baselines.
- Efficiency: WebDreamer is 4–5 times more efficient than tree search in terms of wall-clock time, as it avoids the overhead of loading new pages and backtracking in a live browser.
- Specialization Success: On VisualWebArena, fine-tuning Dreamer-7B with just 25K in-domain instances allowed it to match or surpass GPT-4o's performance, achieving a 23.2% success rate.
Limitations and Future Work
While model-based planning offers significant advantages, it faces a "hallucination wall." As the planning horizon increases beyond 3 steps, error accumulation in the simulation makes deep planning unreliable. Future work will focus on improving the fidelity of multi-step simulations and handling "text-heavy" sites where many small UI changes occur simultaneously. Despite these challenges, WebDreamer demonstrates that specialized world models are a viable path toward efficient and effective web agents.
Summary updated