WebExpert: domain-aware web agents with critic-guided expert experience for high-precision search

Yuelin Hu, Zhengxue Cheng, Ronghua Wu, Qunshan Gu, Hongwei Hu, Wei Liu, Qiao Liang, Li SongarXiv 2026 · arXiv

TL;DR

WebExpert is an end-to-end, domain-aware web browsing agent designed to tackle complex search tasks in specialized fields like finance and biomedicine. By distilling raw QA pairs and expert materials into structured, "schema-light" rules, WebExpert guides large language models to generate highly precise, facet-grounded queries. This approach improves answer accuracy by up to 3.6 percentage points over strong browsing baselines while significantly reducing unnecessary webpage hops.

What Problem Does This Paper Address?

While modern LLM-powered web agents excel at open-ended browsing tasks, their performance plummets when dropped into highly specialized domains like credit approval, clinical guidance, or pharmaceutical research.

Generic agents lack domain priors. When faced with a technical prompt, they often formulate vague or off-target search queries, wander aimlessly through irrelevant webpages, and fail to recognize domain-specific evidence. Human experts, by contrast, instinctively filter their searches through specific contextual dimensions—such as geographic regulations, active policy types, and precise timeframes.

Traditional Retrieval-Augmented Generation (RAG) helps inject knowledge, but it usually happens after the retrieval step. If the agent's initial search queries are poor, RAG simply processes high-quality noise. WebExpert addresses this gap by injecting structured expert experience before and during the query planning phase.

Methods

WebExpert introduces a comprehensive three-step pipeline spanning offline experience construction, supervised fine-tuning, and online execution.

Overview of WebExpert and its three-step pipeline
  1. Critic-Guided Expert Experience Extraction (Offline): The system harvests domain-specific QA pairs and text corpora, normalizes them into canonical intents, and clusters them using multi-view density clustering (UMAP, HDBSCAN, and BERTopic). A contradiction-aware summarizer (powered by DeepSeek-R1) processes these clusters to distill concise rules containing core guidance, edge cases, and known failure modes.
  2. Schema-Light Facet Induction: Instead of forcing engineers to manually write rigid domain lexicons, WebExpert automatically extracts high-frequency domain terms and refines them into flexible facets (e.g., time, region, policy, industry) using weak supervision and corpus statistics.
  3. Experience-Aware Training (SFT): The base model (QwQ-32B) is fine-tuned using a specialized objective function ($L_{plan}$) that up-weights tokens aligning with active domain facets, alongside a contrastive retrieval loss ($L_{ret}$) to ensure the planner prefers high-quality, relevant rules.
  4. Inference with an Experience Gate: During live browsing, the agent retrieves the top-$k$ expert experiences. An "experience gate" calculates the average cosine similarity of these rules. If confidence is high, it biases decoding toward the active facets; if it falls below a calibrated threshold ($\theta = 0.3$), it safely falls back to generic query generation to avoid over-constraining the search.

Datasets

The authors evaluated WebExpert on four prominent benchmarks, utilizing both their open-domain and domain-specific subsets:

  • GAIA: A general AI assistant benchmark testing complex, multi-modal web tasks.
  • GPQA: A graduate-level, "Google-proof" question-answering dataset.
  • HLE (Humanity's Last Exam): A highly challenging dataset designed to push the limits of advanced reasoning models.
  • WebWalkerQA: A benchmark explicitly built for multi-step web browsing, requiring deep page navigation and structured evidence citation across diverse real-world domains.

Key Insights and Ideas

  • Query Grounding vs. Document Reading: The core "aha moment" is that structuring knowledge to guide query formulation is far more effective for web agents than simply appending documents to a prompt. Grounding the query prevents the agent from getting lost in the first place.
  • The Power of the Experience Gate: Knowing when not to use expert rules is just as vital as knowing when to use them. The confidence-based fallback mechanism prevents the agent from becoming bottlenecked by low-quality or irrelevant rules.
  • Moving Beyond Rigid Schemas: By automatically bootstrapping facet vocabularies from weak supervision, the architecture remains highly adaptable to entirely new domains without requiring months of manual ontology engineering.

Key Results

WebExpert consistently outperformed state-of-the-art browsing baselines across all evaluated benchmarks.

Table 2: Main experimental results across benchmarks
  • Accuracy Gains: Combined with Supervised Fine-Tuning (WebExpert+SFT), the model achieved an Answer Exact Match (EM) improvement of 1.5 to 3.6 percentage points over the strongest baseline (WebThinker-32B-Base).
  • Search Efficiency: The agent became significantly more efficient. Average page hops dropped from 8.1 to 5.2, proving that the domain-grounded queries led the agent directly to the right sources with minimal wandering.
  • Query Precision: Query Precision at 3 (QP@3)—the proportion of generated queries that successfully retrieve on-topic evidence—surged from 49.3% (WebThinker) to 61.8% (WebExpert+SFT).

An example of how raw QA data is successfully distilled into these high-precision rules can be seen in the paper's asset diversification case study:

Table 1. From clustered QAs to distilled expert experience: asset correlation and diversification (GPQA - Finance).

Limitations and Future Work

While WebExpert marks a major step forward for specialized web agents, a few limitations remain:

  • Dependency on Seed Data: The offline experience construction phase still relies on having access to a high-quality initial pool of domain-specific QA pairs or curated expert documents.
  • Weak Supervision Requirements: Although it eliminates hand-written lexicons, the schema-light facet induction still requires initial corpus statistics and shallow taggers to bootstrap its categories effectively.

The authors note that the experience base is maintained as a versioned store to support streaming updates, suggesting that future work will likely focus on making the continuous ingestion of real-time web evidence entirely autonomous.

Summary updated

Abstract

Specialized web tasks in finance, biomedicine, and pharmaceuticals remain challenging due to missing domain priors: queries drift, evidence is noisy, and reasoning is brittle. We present WebExpert, a domain-aware web agent that we implement end-to-end, featuring : (i) sentence-level experience retrieval with topic merging and rule distillation, (ii) schemalight facet induction that bootstraps time,region,policy,industry facets from weak supervision instead of static hand-written lexicons, and (iii) preference-optimized planning that jointly improves query planning and retrieval via pairwise preference learning alongside a coverage-aware objective. At inference, a lightweight experience gate biases decoding toward active facets with fallback under low-retrieval confidence. On GAIA, GPQA, HLE, and WebWalkerQA, WebExpert improves Answer Exact Match (EM) by 1.5-3.6 pp over the strongest browsing baseline and reduces page hops. Analysis shows consistent gains and ablations on retrieval, topic merging, facet induction, and preference-aware training.
BibTeX
@misc{hu2026webexpert,
  author = {Yuelin Hu and Zhengxue Cheng and Ronghua Wu and Qunshan Gu and Hongwei Hu and Wei Liu and Qiao Liang and Li Song},
  title = {{WebExpert: domain-aware web agents with critic-guided expert experience for high-precision search}},
  year = {2026},
  eprint = {2604.06177},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2604.06177},
}
Figure