Voluntary Servitude: Weaponized Safety Warning Mechanisms for Controlling AI Agents

Ilan Zendel, Matan Levy, Stav Cohen, Avi Mendelson, Amit LeViarXiv 2026 · arXiv

TL;DR

This paper introduces Voluntary Servitude, a novel attack that manipulates LLM agents by weaponizing their own safety alignment. Instead of using direct commands, attackers plant fabricated safety hazards (like fake reviews claiming a provider is "unsafe") to trick agents into disqualifying legitimate options and selecting the attacker's target, bypassing current injection detectors with up to 4.4x higher effectiveness.

What Problem Does This Paper Address?

As LLM agents move from simple chatbots to autonomous assistants that browse the web, book travel, and manage software dependencies, they increasingly rely on third-party content. The current security paradigm focuses on indirect prompt injection, where an adversary embeds a hidden command (e.g., "Ignore all previous instructions and do X").

Consequently, modern defenses are "instruction-centric," using detectors trained to spot command-like syntax. The authors identify a critical gap: agents can be steered without any commands at all. By providing "facts" that trigger an agent’s internal harm-avoidance logic, an attacker can exploit the very safety guardrails meant to protect the user.

Methods

The researchers developed three "attack families" based on different psychological levers:

  1. Safety: Fabricated hazards (e.g., "The guide abandoned our child during an emergency"). This relies on the model's harm-avoidance training.
  2. Authority: Fake institutional endorsements (e.g., "Ministry of Tourism certified"). This relies on deference to institutions.
  3. Attribute: Superior service quality claims (e.g., "The guide has a PhD"). This relies on ordinary persuasion.
Figure 1: Voluntary servitude steers agents past injection detectors.

The team evaluated these attacks across six different LLMs (including Gemini 3.1, Claude 4.5, and Llama 3.3) and four production browser agents (Edge, Perplexity, Brave, and OpenAI). They measured the Attack Success Rate (ASR) and the Pass Rate through commercial injection detectors like ProtectAI and Lakera Guard.

Datasets

The study utilized a newly constructed benchmark consisting of:

  • 10 Consumer-Service Domains: Ranging from scuba diving centers to boutique wineries.
  • 500 Traveler-Phrased Queries: Realistic user requests for recommendations.
  • Poisoned Contexts: Web pages containing a mix of genuine reviews and the attacker's fabricated payloads.
  • Baselines: They compared their results against standard instruction injection and "misconduct" complaints (ethical issues rather than safety hazards).

Key Insights and Ideas

  • Safety as a Vulnerability: The most striking finding is that safety alignment—the process of training models to avoid harm—can be turned into a steering mechanism. When an agent reads a safety hazard, it doesn't just "dislike" a provider; it eliminates them from consideration entirely.
  • Reasoning vs. Syntax: Because these attacks look like ordinary testimony between humans, they are invisible to detectors looking for "system prompt" overrides or command syntax.
  • Elimination is the Winning Strategy: Safety-based attacks are uniquely powerful because they trigger a binary "unsafe/safe" logic. While attribute-based attacks try to make a target look better, safety attacks make all competitors look unusable.
Figure 4: Share of successes reached by elimination, per attack family and model.

Key Results

  • Effectiveness: Safety payloads steered agents roughly as often as instruction injections (60.4% vs. 66.0%) when no filter was present.
  • Stealth: Behind commercial detectors, the safety attack was 1.7x to 4.4x more effective than instruction injection because the detectors flagged the commands but let the "reviews" pass.
  • Transferability: The attack successfully transferred to production browser agents on live web pages, with some agents (like Microsoft Edge and Brave) selecting the attacker's target in 100% of trials.
  • Defense: The authors proposed a "corroboration requirement"—a simple system prompt instruction telling the agent to treat safety claims as unverified unless supported by multiple independent sources. This lowered the attack success rate from 68.4% to 3.6% without requiring model retraining.

Limitations and Future Work

The authors acknowledge several limitations:

  • Non-Adaptive Attackers: The study focused on attackers who don't change their strategy based on the defense. An "adaptive" attacker might try to manufacture fake corroboration across multiple accounts.
  • Domain Specificity: The evaluation was centered on consumer travel services; extending this to software dependency selection or multi-turn agent sessions is a necessary next step.
  • Alignment Trade-offs: The corroboration defense might cause agents to ignore genuine warnings from single sources, a trade-off that needs further empirical study.

Future work should investigate which specific stage of post-training (SFT vs. RLHF) instills this susceptibility to reported hazards.

Summary updated

Abstract

LLM agents increasingly act on information retrieved from public sources they treat as credible, which turns a manipulated recommendation into a transaction, a dependency, or a decision with physical consequences. The security literature models this threat as instruction injection, in which an adversary writes a command into retrieved content and deployed defenses learn to recognize command syntax. We show that an attacker can instead steer an agent through the agent's own harm-avoidance. A single fabricated hazard, written as an ordinary review, leads the agent to disqualify the legitimate options and select the attacker's target by its own reasoning, with no command for a detector to recognize. We call this voluntary servitude. These payloads steer agents about as often as instruction injection (60.4% vs. 66.0%), and deployed injection detectors score them as benign, so behind a filter they are 1.7-4.4x more effective. We measure this at the generation stage across six models and ten consumer-service domains, for three attack families, and show that it transfers to production browser agents, outperforming instruction injection, and to a retrieval pipeline (1.0-57.0% end-to-end success, depending on the retriever). We then show that a corroboration requirement added to the agent's instructions removes roughly 95% of the exposure against a non-adaptive attacker without retraining. With the requirement in place, the agent still reads the planted hazard and no longer acts on it.
BibTeX
@misc{zendel2026voluntary,
  author = {Ilan Zendel and Matan Levy and Stav Cohen and Avi Mendelson and Amit LeVi},
  title = {{Voluntary Servitude: Weaponized Safety Warning Mechanisms for Controlling AI Agents}},
  year = {2026},
  eprint = {2611.04393},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.04393},
}
Figure