Abstract
Retrieval-augmented generation (RAG) systems enhance large language models by incorporating external knowledge, addressing issues like outdated internal knowledge and hallucination. However, their reliance on external knowledge bases makes them vulnerable to corpus poisoning attacks, where adversarial passages can be injected to manipulate retrieval results. Existing methods for crafting such passages, such as random token replacement or training inversion models, are often slow and computationally expensive, requiring either access to retriever’s gradients or large computational resources. To address these limitations, we propose Dynamic Importance-Guided Genetic Algorithm (DIGA), an efficient black-box method that leverages two key properties of retrievers: insensitivity to token order and bias towards influential tokens. By focusing on these characteristics, DIGA dynamically adjusts its genetic operations to generate effective adversarial passages with significantly reduced time and memory usage. Our experimental evaluation shows that DIGA achieves superior efficiency and scalability compared to existing methods, while maintaining comparable or better attack success rates across multiple datasets.
Introduction
Retrieval-Augmented Generation (RAG) has become a standard architecture for deploying Large Language Models (LLMs) in production. By retrieving relevant documents from an external corpus to ground the LLM's responses, RAG significantly reduces hallucinations and allows models to access up-to-date information. However, this dependency introduces a critical security vulnerability: corpus poisoning.
In a corpus poisoning attack, an adversary injects a small number of carefully crafted "adversarial passages" into the knowledge base. These passages are designed to be highly relevant to a wide range of user queries, ensuring they are retrieved and subsequently influence the LLM's output—potentially leading to the dissemination of misinformation or malicious instructions.
Current attack methodologies face significant hurdles. White-box attacks like HotFlip require access to the retriever's internal gradients, which is impractical for most commercial APIs. Conversely, black-box inversion-based methods like Vec2Text require training expensive auxiliary models that often fail to generalize across different domains.
The DIGA Framework
To overcome these challenges, we introduce DIGA (Dynamic Importance-Guided Genetic Algorithm), presented at NAACL 2025. DIGA is a training-free, black-box optimization method that treats corpus poisoning as an evolutionary search problem. Its design is rooted in two fundamental observations regarding the behavior of modern dense retrievers:
- Insensitivity to Order: Retrievers often behave like sophisticated "bag-of-words" models, where the specific arrangement of tokens has a negligible impact on the final embedding.
- Bias to Influential Tokens: A small subset of "influential" tokens (typically domain-specific keywords) contributes disproportionately to the document's representation in the embedding space.
The DIGA Pipeline
DIGA optimizes adversarial passages through a multi-stage evolutionary process:
- Token Scoring: To avoid the computational cost of iterative testing, DIGA uses TF-IDF as an efficient proxy to identify influential tokens within the target dataset.
- Population Initialization: The algorithm begins with a set of candidate passages heavily biased toward these high-importance tokens.
- Evolutionary Loop: Through successive generations, the algorithm performs selection (keeping passages closest to target query centroids), crossover (swapping segments between candidates), and Dynamic Mutation. This mutation strategy is the core innovation: it prioritizes the replacement of low-importance tokens while preserving influential ones, accelerating convergence.
- Hybrid Fine-Tuning: The final phase utilizes a standard genetic algorithm to refine the passage for maximum retrieval performance.
Experimental Evaluation
We evaluated DIGA across four diverse datasets from the BEIR benchmark: Natural Questions (NQ), NFCorpus (medical), FiQA (finance), and SciDocs (scientific).
Efficiency and Scalability
One of DIGA's primary advantages is its efficiency. Compared to the white-box HotFlip method, DIGA is 6.5x to 9.3x faster and requires 6x to 10.5x less GPU memory. This allows the attack to be executed on consumer-grade hardware, such as a single RTX 3090, whereas previous methods often required enterprise-level resources. Furthermore, DIGA scales effectively; as more adversarial passages are injected, the Attack Success Rate (ASR) increases consistently.
Attack Success Rate (ASR)
In black-box scenarios, DIGA consistently outperforms existing state-of-the-art methods. For instance, on the NFCorpus dataset, DIGA achieved an ASR@20 of 44.3%, surpassing Vec2Text's 38.7%. While white-box attacks still maintain a performance edge due to their access to model gradients, DIGA significantly narrows the gap for practical, black-box applications.
Transferability and Downstream Impact
Adversarial passages generated by DIGA exhibit strong transferability. Because they focus on influential keywords, they often remain effective even when the victim switches to a different retriever architecture, including sparse retrievers like BM25.
To demonstrate the real-world risk, we tested the impact on downstream LLMs. In experiments with GPT-3.5, DIGA-generated passages successfully induced "indirect prompt injection," causing the model to ignore user instructions and output a target "I don't know" response in 67% of cases when the adversarial document was retrieved.
Conclusion
DIGA exposes a fundamental "laziness" in modern dense retrievers, demonstrating that effective corpus poisoning does not require complex model inversion or gradient access. By exploiting token-level biases and order insensitivity, DIGA provides a highly efficient and scalable tool for auditing the security of RAG systems. This work underscores the urgent need for "retriever-hardening" techniques that can resist such keyword-centric adversarial manipulations.
Summary updated