TL;DR
WebHarbor introduces a suite of 153 dockerized, executable website environments designed to provide the stability of synthetic benchmarks with the visual and functional depth of the real web. By using a human-agent collaborative pipeline to "dock" real-world services, the authors enable reliable evaluation and efficient reinforcement learning (RL) for GUI agents, particularly for complex, authenticated workflows that are often blocked or broken on the live web.
What Problem Does This Paper Address?
Building agents that can navigate the web like humans is a major goal in AI, but the "training ground" is currently broken. Researchers face a frustrating trade-off:
- Live Web Benchmarks: These are realistic but highly unstable. Websites change layouts, implement bot detection (like CAPTCHAs), or have regional restrictions. The authors found that ~38% of tasks in some benchmarks are now blocked by bot detection, making reproducible research nearly impossible.
- Synthetic Environments: These are stable and reproducible but often "uncanny" or shallow. They lack the visual complexity, merchandising hierarchies, and functional depth (like actual database-backed transactions) of real sites.
Furthermore, most existing benchmarks ignore authenticated workflows—the things we actually use the web for, like booking a hotel or managing an account—because they are difficult to simulate safely and repeatedly.
Methods
The authors propose a process called Task-Driven Docking. Instead of trying to reconstruct an entire commercial site (which is impractical), they focus on the specific pages, data, and interactions required to complete a set of target tasks.
The construction follows a human-agent collaborative pipeline:
- Coding Agents: These agents implement the frontend (Jinja2 templates), backend (Flask), and database (SQLite) for the environment. They use "reusable skills" to crawl a target site and reconstruct its logic.
- Human-in-the-Loop: Human contributors propose tasks and, crucially, audit the results. They check for visual fidelity, functional correctness (do the buttons actually work?), and "task validity" (is the answer leaked in the UI?).
- Evolution: As new tasks are added, the environment evolves. If a task requires a "Watch Later" feature on a YouTube clone, the coding agent implements that specific backend logic.
The result is a Dockerized environment that runs entirely offline, supports deterministic resets to a seeded state, and provides high-fidelity visual observations.
Datasets
WebHarbor consists of:
- 153 Environments: Reconstructed from sites found in popular benchmarks like WebVoyager and Online-Mind2Web.
- 3,518 Tasks: Spanning information retrieval, account management, and transactions.
- Train/Test Split: 123 sites are used for training, while 30 sites (including all 15 from WebVoyager) are held out for testing to evaluate generalization.
Key Insights and Ideas
- Fidelity is the Bottleneck for RL: Standard synthetic environments often simplify layouts too much. Agents trained on "clean" synthetic data often fail when faced with the visual noise and complex hierarchies of the real web. WebHarbor's focus on "target-like" navigation helps bridge this gap.
- The "Docking" Philosophy: By replacing live dependencies with local assets and seeded states, the authors create a "sandbox" where agents can fail, learn, and reset thousands of times per hour without being banned or incurring high costs.
- Human Auditing is Non-Negotiable: The authors found that fully automated reconstructions often had "shallow" interactions (e.g., buttons that look right but don't update the database). Human reviewers were essential to ensure the environments weren't just "visual shells" but functional replicas.
- Generalization to the Live Web: A key finding is that training in these "docked" environments actually improves performance on the real live web, proving that the learned policies are not just over-fitting to the simulator.
Key Results
The authors trained a Qwen 4B-level agent using Reinforcement Learning (specifically the MM-GRPO objective) entirely within WebHarbor.
- Superior Performance: The resulting model, WebHarbor-4B, achieved a live-web average score of 75.8, outperforming the SFT baseline by 22.2 points and even beating OpenWebRL-4B (a SOTA agent trained on live websites) by 5.2 points.
- Authenticated Workflow Gains: The agent showed massive improvements in tasks requiring login and state changes. On the WebArena benchmark, it led on four out of five sites, demonstrating that it had learned the "logic" of authentication and multi-step transactions.
- Ranking Consistency: There is a high correlation (Spearman $\rho=1.0$) between model rankings on WebHarbor and the live WebVoyager benchmark, suggesting WebHarbor is a reliable proxy for real-world performance.
Limitations and Future Work
- Imitation vs. Reality: WebHarbor is still an imitation. Highly dynamic, client-heavy behaviors (like interactive maps or real-time stock tickers) are often replaced with static surrogates.
- Judge Reliability: While they use "Agent-as-a-Judge" and scripted verifiers, evaluating open-ended web tasks remains difficult. Judges can sometimes miss subtle failures or overlook valid alternative paths.
- Scaling: While the human-agent pipeline is more scalable than manual coding, it still requires significant community effort (57 contributors over 5 months for this version).
- Model Scale: Due to compute constraints, the authors only trained a 4B parameter model. Testing how larger frontier models (like 70B+ parameters) benefit from this environment is a clear next step.
Summary updated