TL;DR
This paper demonstrates how attackers can exploit vulnerabilities in LLM-based web navigation agents through "Indirect Prompt Injection" (IPI) attacks. By embedding malicious code into webpages, they can hijack these agents to perform unintended actions, such as exfiltrating login credentials or forcing ad clicks, posing a significant security risk.
What Problem Does This Paper Address?
Large Language Model (LLM)-powered autonomous web navigation agents are rapidly becoming more sophisticated, capable of performing complex tasks online like booking reservations or managing finances. However, their reliance on LLMs makes them susceptible to prompt injection attacks. While direct prompt injection is understood, this paper focuses on Indirect Prompt Injection (IPI), where malicious instructions are hidden in external resources (like webpages) that the agent retrieves. The core problem is that these agents, by design, translate LLM outputs into browser actions, making them vulnerable to being manipulated by hidden instructions without the user's explicit knowledge or consent. This research aims to demonstrate the practical realization of IPI attacks on these agents and highlight the critical security risks they introduce.
Methods
The researchers leverage the Greedy Coordinate Gradient (GCG) algorithm, originally designed for finding universal adversarial triggers for LLMs, to craft malicious payloads. They adapt GCG to find "universal adversarial triggers" that can reliably hijack LLM-based web navigation agents across different prompts and contexts.
The core methodology involves:
- Browser Gym Framework: They use Browser Gym, an open-source framework for creating LLM-based web navigation agents, powered by Meta's Llama-3.1-8B-Instruct model. Browser Gym extracts the webpage's accessibility tree (parsed HTML) and compiles it into a prompt for the LLM, which then selects actions (click, scroll, etc.).
- GCG Algorithm Adaptation: The GCG algorithm is modified to optimize triggers not for a single prompt, but for a set of different prompts and contexts derived from webpage HTML. This allows for the creation of "universal" triggers that are effective across various user instructions and website elements.
- Targeted and Universal Attacks: They demonstrate two main attack scenarios:
- Targeted Website Targeted Instruction (TWTI): Optimizing a trigger for a specific website and a specific user instruction.
- Targeted Website Universal Instruction (TWUI): Optimizing a trigger that works universally across a group of websites for a specific type of instruction (e.g., stealing login credentials).
- Stealth Techniques: To make attacks less detectable, they explore hiding triggers within URLs, using transparent fonts, or hiding elements with CSS.
Datasets
The paper does not introduce new datasets for training LLMs. Instead, it uses five real-world websites (Chess.com, games.penjee.com, citybrewtours.com, norway.no, and translate.google.com) as targets for their attack demonstrations. They also create a benchmark of 200 various navigation goals for each of the five sample websites to evaluate the attack success rate of their universal triggers. For the Universal Website Targeted Instruction (UWTI) scenario, they use a training set of eight real-world login pages and a test set of eleven other login pages.
Key Insights and Ideas
- IPI is a Practical Threat: The research moves beyond theoretical concerns, providing concrete demonstrations of how IPI attacks can be executed against real-world LLM web agents on actual websites.
- Universal Triggers are Powerful: The GCG algorithm can be adapted to find "universal" adversarial triggers that can hijack agent behavior across different user instructions and website contexts, making attacks more scalable.
- Stealth is Achievable: Attackers can employ techniques like URL embedding or CSS manipulation to hide malicious triggers, making them difficult for users and potentially even defenses to detect.
- Security Risks are Diverse: The attacks can lead to a range of malicious outcomes, including data exfiltration (login credentials), forced advertisement clicks, and redirection to malicious sites.
- Optimization Time is Manageable: The optimization process for finding triggers, while computationally intensive, can be significantly shortened by using smaller search widths and including the target string in the initial trigger.
Key Results
- High Attack Success Rates: The researchers achieved high attack success rates (ASR) across their sample websites, with the lowest observed ASR being 0.83. In some cases, they were able to exfiltrate login credentials with an ASR of 0.55 on the test dataset.
- Reduced Optimization Time: By using a search width of 128, the average runtime for trigger optimization was reduced to under three hours. Including the target sequence in the initial trigger further reduced this to under an hour, sometimes even under ten minutes.
- Trigger Length and Loss Function Insignificance: Increasing trigger length or using the Carlini-Wagner (CW) loss function did not significantly improve optimization time-to-completion compared to standard cross-entropy loss.
- Limited Transferability: Triggers optimized for one LLM (Llama-3.1) were not directly transferable to other LLMs (Llama-2, Mistral-7B-Instruct), suggesting model-specific tuning might be required.
Limitations and Future Work
- Model Specificity: The current triggers are optimized for a particular LLM (Llama-3.1). Agents using different LLMs are less susceptible to these specific triggers. Future work could explore joint optimization across multiple models.
- HTML Access Requirement: The attacker needs access to some part of the HTML that the navigation agent will consume. This limits attacks to scenarios where the attacker can inject content into a webpage or its associated resources.
- Target Sequence Validity: Triggers are optimized for specific target sequences that must be syntactically valid within the agent's action framework.
- Failure Analysis: The paper identifies categories of failed attacks, often related to high prior probabilities for correct actions or reasoning phrases in the LLM's output. This suggests that defenses might focus on mitigating these inherent LLM biases.
- Future Work: The authors suggest exploring more sophisticated defenses, such as decoupling planning from execution or observing action dependencies on instructions, to combat IPI attacks. They also highlight the need for robust defenses as LLM-enabled web navigation agents become more prevalent.
Summary updated