Dark Patterns Meet GUI Agents: LLM Agent Susceptibility to Manipulative Interfaces and the Role of Human Oversight

Jingyu Tang, Chaoran Chen, Jiawen Li, Zhiping Zhang, Bingcan Guo, Ibrahim Khalilov, Simret Araya Gebreegziabher, Bingsheng Yao, Dakuo Wang, Yanfang Ye, Tianshi Li, Ziang Xiao, Yaxing Yao, Toby Jia-Jun LiarXiv 2025 · OpenReview

Abstract: The dark patterns, deceptive interface designs manipulating user behaviors, have been extensively studied for their effects on human decision-making and autonomy. Yet, with the rising prominence of LLM-powered GUI agents that automate tasks from high-level intents, understanding how dark patterns affect agents is increasingly important. We present a two-phase empirical study examining how agents, human participants, and human-AI teams respond to 16 types of dark patterns across diverse scenarios. Phase 1 highlights that agents often fail to recognize dark patterns, and even when aware, prioritize task completion over protective action. Phase 2 revealed divergent failure modes: humans succumb due to cognitive shortcuts and habitual compliance, while agents falter from procedural blind spots. Human oversight improved avoidance but introduced costs such as attentional tunneling and cognitive load. Our findings show neither humans nor agents are uniformly resilient, and collaboration introduces new vulnerabilities, suggesting design needs for transparency, adjustable autonomy, and oversight.

Introduction

For years, Human-Computer Interaction (HCI) researchers have studied "dark patterns"—UI tricks like pre-checked boxes for data sharing or hidden fees—and their impact on human autonomy. However, we are entering an era where we delegate tasks to GUI agents (like OpenAI’s Operator or Claude’s Computer Use) that navigate the web for us. This research, conducted by the University of Notre Dame and other institutions, investigates a critical gap: whether these agents are resilient to the same deceptions that plague humans.

Methodology

The researchers conducted a two-phase empirical study to evaluate agent and human-AI team resilience:

  • Phase 1 (Agent Susceptibility): Six GUI agents—four "scaffolded" agents using the Browser Use framework (GPT-4o, Claude 3.7) and two "end-to-end" commercial agents (OpenAI Operator, Claude Computer Use)—were evaluated for awareness (internal reasoning traces) and avoidance (successful bypass of traps).
  • Phase 2 (Human-AI Teaming): A within-subjects study with 22 participants compared three conditions: humans working alone, agents working alone, and humans supervising an agent in a "watch mode."

The study utilized a reusable evaluation pipeline with 16 dark pattern types (e.g., Trick Questions, Bad Defaults, Confirmshaming) embedded in custom-built React websites simulating e-commerce and social media environments.

Figure 1: An overview of a GUI agent interacting with a 'Trick Question' dark pattern during a subscription task. The agent prioritizes checking all boxes to complete the task, failing to recognize the manipulative language.

Key Insights: Goal-Driven Myopia and Procedural Blind Spots

A primary finding of the study is Goal-Driven Myopia. Agents often recognized a dark pattern in their reasoning traces but proceeded anyway, prioritizing the user's high-level instruction (e.g., "subscribe") over protective "common sense." In "Bad Default" tasks, agents often noted pre-checked boxes but refused to uncheck them, viewing the extra step as unnecessary for the primary goal.

Furthermore, the study identified Divergent Failure Modes. Humans typically fail due to "System 1" thinking—scanning quickly and relying on habits. In contrast, agents fail due to "procedural blind spots," lacking the social and normative context to realize that a "Trick Question" is a trap designed to deceive.

Figure 2: A detailed view of the 'Trick Question' dark pattern, where the agent must navigate confusing language to avoid unwanted data sharing.

The Oversight Paradox

While human oversight significantly boosted avoidance rates for certain patterns (e.g., "Bad Defaults" avoidance rose from 33.3% to 80%), it introduced the Oversight Paradox. Supervising an agent often led to "attentional tunneling," where humans stopped looking at the whole page and only focused on the agent's specific path. Consequently, 66% of participants in the oversight condition failed to report awareness of dark patterns, compared to 59% when working alone.

Conclusion

The study concludes that neither humans nor agents are uniformly resilient to manipulative interfaces. The authors suggest moving toward "adaptive autonomy," where agents proactively pause and ask for clarification when they detect potential manipulative intent. Additionally, the paper calls for updating consumer protection laws to account for "agent-mediated" deception, as current regulations focus almost entirely on direct human-to-interface interaction.

Summary updated

Abstract

The dark patterns, deceptive interface designs manipulating user behaviors, have been extensively studied for their effects on human decision-making and autonomy. Yet, with the rising prominence of LLM-powered GUI agents that automate tasks from high-level intents, understanding how dark patterns affect agents is increasingly important. We present a two-phase empirical study examining how agents, human participants, and human-AI teams respond to 16 types of dark patterns across diverse scenarios. Phase 1 highlights that agents often fail to recognize dark patterns, and even when aware, prioritize task completion over protective action. Phase 2 revealed divergent failure modes: humans succumb due to cognitive shortcuts and habitual compliance, while agents falter from procedural blind spots. Human oversight improved avoidance but introduced costs such as attentional tunneling and cognitive load. Our findings show neither humans nor agents are uniformly resilient, and collaboration introduces new vulnerabilities, suggesting design needs for transparency, adjustable autonomy, and oversight.
BibTeX
@misc{tang2025dark,
  author = {Jingyu Tang and Chaoran Chen and Jiawen Li and Zhiping Zhang and Bingcan Guo and Ibrahim Khalilov and Simret Araya Gebreegziabher and Bingsheng Yao and Dakuo Wang and Yanfang Ye and Tianshi Li and Ziang Xiao and Yaxing Yao and Toby Jia-Jun Li},
  title = {{Dark Patterns Meet GUI Agents: LLM Agent Susceptibility to Manipulative Interfaces and the Role of Human Oversight}},
  year = {2025},
  eprint = {2509.10723},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2509.10723},
}

Loading related content...

Figure