FocusAgent: Simple Yet Effective Ways of Trimming the Large Context of Web Agents

Imene Kerboua, Sahar Omidi Shayegan, Megh Thakkar, Xing Han Lù, Léo Boisvert, Massimo Caccia, Jérémy Espinas, Alexandre Aussem, Véronique Eglin, Alexandre Lacoste

TL;DR

FocusAgent introduces a simple, two-stage framework that uses a lightweight LLM as a dynamic retriever to prune web page accessibility trees (AxTrees) before passing them to a web agent. This method reduces observation context sizes by over 50% and slashes token costs while maintaining strong task performance. Crucially, this pruning step doubles as a security sanitizer, dropping the success rate of malicious prompt-injection attacks from over 80% to under 1%.

What Problem Does This Paper Address?

Large Language Model (LLM) web agents are increasingly used to automate complex, multi-step tasks on the internet. However, modern web pages are massive. Even when compressed from raw HTML/DOM into an Accessibility Tree (AxTree)—which represents the page's user interface elements—the text can easily span tens of thousands of tokens.

This massive context window creates three major bottlenecks:

  1. High Computational Cost: Processing massive inputs at every single navigation step quickly racks up expensive API bills.
  2. Context Saturation: LLMs can get distracted or overwhelmed by noisy, irrelevant information on a page, leading to poor action planning.
  3. Security Vulnerabilities: Long pages expose agents to indirect prompt injection attacks, where malicious text hidden in ads, banners, or pop-ups hijacks the agent's instructions to steal data or perform unauthorized actions.

Existing solutions rely on basic truncation (cutting off the bottom of the page) or standard embedding-based vector search. Truncation frequently deletes critical information, while embedding models struggle because web navigation requires an understanding of dynamic state and planning context, not just static semantic similarity to the user's initial goal.

Methods

FocusAgent solves this with a two-stage pipeline that decouples page filtering from action execution:

Overview of the FocusAgent pipeline showing Stage 1 (Retrieval) and Stage 2 (Action Prediction)
  • Stage 1: LLM-Based Retrieval: The framework first numbers every line of the raw AxTree. A lightweight, inexpensive LLM (like GPT-4.1-mini) acts as a retriever. Guided by the user's goal, it uses Chain-of-Thought (CoT) reasoning to identify and output the specific line ranges necessary to complete the task.
  • Soft Prompting Strategy: To ensure the retriever doesn't accidentally delete critical elements, the authors employ a "soft" prompting strategy. The retriever is explicitly instructed to prioritize recall—meaning if it is uncertain whether a line is relevant, it should keep it rather than risk task failure.
  • Stage 2: Action Prediction: The irrelevant lines are stripped out, replaced by a simple placeholder indicating how many lines were pruned. This clean, compact AxTree is then passed to the primary backbone agent (e.g., GPT-4.1 or Claude-3.7-Sonnet), which decides on the next action (click, scroll, fill, etc.).

Datasets

The authors evaluated FocusAgent using the BrowserGym ecosystem across two primary web navigation benchmarks:

  • WorkArena L1: A benchmark focused on enterprise knowledge work tasks (e.g., editing forms, sorting lists, ordering items in a service catalog).
  • WebArena: A highly realistic web environment containing tasks spread across various categories like Shopping, Gitlab, Maps, and Reddit.
  • DoomArena: A specialized security framework used to test web agents against adversarial prompt injections, specifically banner and pop-up attacks.

Key Insights and Ideas

  • State and Planning Matter More Than Keywords: Traditional retrievers (like BM25 or embedding models) fail because they only look for text matching the user's goal. FocusAgent succeeds because an LLM retriever can infer that structural elements—like a shopping cart or a navigation sidebar—are necessary for the plan, even if they aren't explicitly mentioned in the goal text.
  • Retriever History is Distracting: Counterintuitively, the authors found that providing the interaction history to the lightweight retriever degraded performance. Smaller models get confused by long interaction histories; they perform better when evaluating the current page state in isolation.
  • Security via Sanitization: By forcing an LLM to actively select only what is relevant to the user's goal, adversarial prompt injections (like fake "ALERT" pop-ups trying to redirect the agent) are naturally ignored and pruned out. The attack text never even reaches the main backbone agent.

Key Results

  • Massive Context Reduction: FocusAgent successfully pruned AxTree observations by 50% to 61% on average across benchmarks.
  • Maintained Performance: On WorkArena L1, FocusAgent paired with a GPT-5-mini retriever achieved a 53.2% success rate, nearly matching the unpruned baseline agent's 53.6%, but at a fraction of the token cost.
Success Rate vs. Average AxTree Token Pruning across different retrieval models
  • Drastic Security Improvements: In pop-up attack scenarios on DoomArena, standard agents suffered from massive vulnerabilities. FocusAgent dropped the Attack Success Rate (ASR) from over 80% down to less than 1% for both GPT-4.1 and Qwen backbone models.
  • The Latency Trade-Off: The primary downside is speed. Because the system requires two sequential LLM calls per step (one to prune, one to act), FocusAgent increased per-step latency by an average of 7.6 seconds.

Limitations and Future Work

The most prominent limitation is the latency overhead introduced by the two-stage design, making it less ideal for real-time applications where sub-second responses are mandatory. Additionally, the system remains sensitive to prompt engineering; if the retriever is too aggressive, it can trap the agent in infinite loops (e.g., scrolling endlessly because an expand button was pruned away).

Future work includes exploring ways to optimize this latency and investigating more sophisticated, adaptive attacks that might attempt to mimic valid structural page elements to bypass the LLM retriever.

Summary updated

Abstract

Web agents powered by large language models (LLMs) must process lengthy web page observations to complete user goals; these pages often exceed tens of thousands of tokens. This saturates context limits and increases computational cost processing; moreover, processing full pages exposes agents to security risks such as prompt injection. Existing pruning strategies either discard relevant content or retain irrelevant context, leading to suboptimal action prediction. We introduce FocusAgent, a simple yet effective approach that leverages a lightweight LLM retriever to extract the most relevant lines from accessibility tree (AxTree) observations, guided by task goals. By pruning noisy and irrelevant content, FocusAgent enables efficient reasoning while reducing vulnerability to injection attacks. Experiments on WorkArena and WebArena benchmarks show that FocusAgent matches the performance of strong baselines, while reducing observation size by over 50%. Furthermore, a variant of FocusAgent significantly reduces the success rate of prompt-injection attacks, including banner and pop-up attacks, while maintaining task success performance in attack-free settings. Our results highlight that targeted LLM-based retrieval is a practical and robust strategy for building web agents that are efficient, effective, and secure.
BibTeX
@misc{kerboua2025focusagent,
  author = {Imene Kerboua and Sahar Omidi Shayegan and Megh Thakkar and Xing Han Lù and Léo Boisvert and Massimo Caccia and Jérémy Espinas and Alexandre Aussem and Véronique Eglin and Alexandre Lacoste},
  title = {{FocusAgent: Simple Yet Effective Ways of Trimming the Large Context of Web Agents}},
  year = {2025},
  eprint = {2510.03204},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2510.03204},
}
Figure