This paper introduces a framework for adversarial self-evolution of attacks against LLM-based search agents. By framing the interaction as a bi-level optimization problem, the authors develop a method called CREO that autonomously evolves sophisticated, stealthy attack strategies that can bypass adaptive defenses and induce "cognitive paralysis" in state-of-the-art models.
TL;DR
The authors propose a framework where adversarial attacks on search agents self-evolve through bi-level optimization, pitting an evolving attacker against an adaptive defender. Their core method, Contrastive Rollout Evolutionary Optimization (CREO), uses feedback from reasoning traces to autonomously generate complex, multi-step poisoning attacks that significantly outperform static, human-crafted baselines.
What Problem Does This Paper Address?
Modern search agents (like those used in RAG systems) perform complex multi-hop reasoning by retrieving external web information. However, this reliance on external data makes them vulnerable to RAG poisoning—the injection of malicious information into the retrieval stream.
The authors identify two major flaws in current attack research:
- Static Strategies: Most attacks rely on human-crafted templates (e.g., simple entity substitution), which are easy to filter and fail to capture the dynamic nature of real-world threats.
- Feedback-Agnosticism: Existing attacks don't adapt to a defender's countermeasures. In the real world, as defenses improve, attackers iterate; current benchmarks don't reflect this "arms race."
Methods
The paper formalizes the security evaluation as a bi-level optimization problem:
- Inner Loop (Defender): The defender updates its "defensive knowledge" based on the attacker's current strategy pool (simulating an informed, adaptive defense).
- Outer Loop (Attacker): The attacker evolves a new strategy pool to maximize the probability of compromising the agent.
To solve the outer loop, they introduce Contrastive Rollout Evolutionary Optimization (CREO). CREO operates in the discrete space of natural language:
- Contrastive Rollouts: The system performs multiple reasoning passes. By comparing successful vs. failed traces (contrastive signals), it identifies specific vulnerabilities.
- Evolutionary Operators: An LLM acts as the "optimizer," performing directional mutation and crossover on attack strategies based on these contrastive signals.
- Diversity Maintenance: A homogeneity constraint ensures the strategy pool doesn't collapse into a single type of attack, maintaining a broad "attack surface."
The authors also built a Deterministic Sandbox to ensure noise-free evaluation and introduced three fine-grained metrics:
- Rigorous Adversarial Accuracy (Acc): Measures if the agent is correct and its reasoning is uncontaminated.
- Adversarial Success Rate (ASR): Measures "logic hijacking," where the agent adopts forged evidence.
- Adversarial Paralysis Rate (APR): A new metric for "denial of service," where the agent detects a contradiction it cannot resolve and refuses to answer.
Datasets
The framework was evaluated on three popular multi-hop QA benchmarks:
- HotpotQA
- 2WikiMultihopQA
- MuSiQue
These were tested against five frontier LLMs: Qwen-Turbo, Qwen3-Max, DeepSeek-V3.2, Kimi-K2.5, and GLM-5.1.
Key Insights and Ideas
- Cognitive Paralysis is a Valid Attack: Traditional metrics only look at whether the final answer is wrong. The authors argue that forcing an agent into a state of "I can't decide because the sources contradict" (APR) is a successful denial-of-service attack.
- Evolutionary Stealth: As strategies evolve, they move from "conflict induction" (injecting obvious lies) to "covert misdirection." For example, a Round 3 strategy might retain real data as a "trust anchor" while only tampering with a single, less salient variable to evade detection.
- The "Consensus Illusion": One particularly effective evolved strategy involves generating a web of interconnected fake documents that all reinforce the same lie, tricking the agent into believing there is a multi-source consensus.
- Diversity Matters: Without the homogeneity constraint, the evolutionary process quickly converges on a few redundant strategies. Maintaining diversity is key to breaking through different types of defensive filters.
Key Results
CREO consistently outperformed static baselines like FactSwap (rule-based) and AuthChain (optimization-based).
- Superior Efficacy: In many cases, CREO reduced agent accuracy (Acc) to near zero. For instance, on MuSiQue with GLM-5.1, accuracy dropped from a base of 82% to just 16% after one round of evolution.
- High Paralysis: Static attacks almost never triggered "Cognitive Paralysis" (APR near 0%). In contrast, CREO's evolved strategies achieved APRs as high as 82% (DeepSeek-V3.2 on HotpotQA).
- Resilience to Defense: Even in the "Sync" setting—where the defender was explicitly told the attacker's strategies—CREO maintained high attack efficacy, proving its ability to find new, unaddressed vulnerabilities.
Limitations and Future Work
- Fixed Retrieval Structures: The current work focuses on multi-hop QA, which has relatively predictable retrieval patterns. The authors note that more complex "Deep Research" agents involving dynamic tool invocation and long-chain reasoning remain an open challenge.
- Computational Cost: The framework requires significant LLM API calls to perform rollouts and drive the evolutionary optimizer, making it more expensive than static attack methods.
- Defense Development: While the paper focuses on the attack side to expose vulnerabilities, the authors emphasize the urgent need for researchers to develop adaptive defense mechanisms that can counter such self-evolving threats.
Summary updated