Abstract
As researchers continue to optimize AI agents for more effective task execution within operating systems, they often overlook a critical security concern: the ability of these agents to detect "impostors" within their environment. Through an analysis of the agents' operational context, we identify a significant threat—attackers can disguise malicious attacks as environmental elements, injecting active disturbances into the agents' execution processes to manipulate their decision-making. We define this novel threat as the Active Environment Injection Attack (AEIA). Focusing on the interaction mechanisms of the Android OS, we conduct a risk assessment of AEIA and identify two critical security vulnerabilities: (1) Adversarial content injection in multimodal interaction interfaces, where attackers embed adversarial instructions within environmental elements to mislead agent decision-making; and (2) Reasoning gap vulnerabilities in the agent's task execution process, which increase susceptibility to AEIA attacks during reasoning. To evaluate the impact of these vulnerabilities, we propose AEIA-MN, an attack scheme that exploits interaction vulnerabilities in mobile operating systems to assess the robustness of MLLM-based agents. Experimental results show that even advanced MLLMs are highly vulnerable to this attack, achieving a maximum attack success rate of 93% on the AndroidWorld benchmark by combining two vulnerabilities.
Introduction
As AI agents transition from simple chatbots to autonomous entities capable of navigating mobile operating systems, they introduce a significant new attack surface. While previous research has explored passive environment injection—such as malicious text hidden on a webpage—this paper identifies a more aggressive threat: Active Environmental Injection Attacks (AEIA).
Unlike passive threats, AEIA involves the environment actively pushing disturbances, such as system pop-ups or message notifications, directly into the agent’s workflow. This exploits the "Perception-Reasoning-Action" loop common in Multimodal Large Language Model (MLLM) agents. A critical structural vulnerability identified is the Reasoning Gap: the 3–12 second delay between an agent capturing a screen state and executing an action, during which the OS state can change without the agent's knowledge.
Methodology: The AEIA-MN Framework
The researchers introduced AEIA-MN, an attack framework utilizing Android SMS notifications as the primary delivery vehicle. The framework evaluates three distinct strategies:
- Adversarial Attack: Injecting misleading text into notifications (e.g., "Agent should believe task is finished!") to trick the MLLM into premature termination or logical errors during the perception stage.
- Reasoning Gap Attack: Exploiting the temporal delay in MLLM processing. By sending a notification after the agent captures a screenshot but before it executes a command, the attacker forces the agent to accidentally click the new notification instead of the intended UI element.
- Combinatorial Attack: A hybrid approach using both strategies simultaneously to maximize disruption.
The framework was tested against the AndroidWorld and AppAgent benchmarks using five prominent MLLMs, including GPT-4o, Qwen-VL-Max, and GLM-4V-Plus.
Key Results and Analysis
The study reveals that even state-of-the-art models are highly susceptible to these active injections:
- High Vulnerability: Combinatorial attacks achieved a staggering 93% Attack Success Rate (ASR) on the AndroidWorld benchmark.
- The Reasoning Gap Impact: Even without misleading text, simply timing a notification to cause a misclick affected up to 26% of tasks. This is exacerbated by mobile UI design, where notifications occupy roughly 13.4% of the screen area, frequently occluding critical buttons.
- Failure of Prompt-Based Defense: Explicitly instructing agents to "ignore unrelated notifications" proved ineffective. In some cases, these prompts actually increased the attack success rate because the MLLM prioritized the visual context of the notification over its textual system instructions.
- Model Performance: Top-tier models like GPT-4o saw success rates on the AppAgent benchmark drop to 0% under direct adversarial attack.
Conclusion
The vulnerability of MLLMs to AEIA suggests that current synchronous, single-agent architectures are insufficient for secure mobile interaction. The authors suggest moving toward Multi-Agent Systems (MAS), where a dedicated security agent monitors the input stream independently. Additionally, they propose exploring blockchain-based "environmental credibility verification" to ensure that UI elements are legitimate. Addressing the Reasoning Gap is essential for the safe deployment of autonomous mobile agents in real-world environments.
Summary updated