Abstract
Retrieval-augmented generation (RAG) systems enhance large language models by incorporating external knowledge, addressing issues like outdated internal knowledge and hallucination. However, their reliance on external knowledge bases makes them vulnerable to corpus poisoning attacks, where adversarial passages can be injected to manipulate retrieval results. Existing methods for crafting such passages, such as random token replacement or training inversion models, are often slow and computationally expensive, requiring either access to retriever's gradients or large computational resources. To address these limitations, we propose Dynamic Importance-Guided Genetic Algorithm (DIGA), an efficient black-box method that leverages two key properties of retrievers: insensitivity to token order and bias towards influential tokens. By focusing on these characteristics, DIGA dynamically adjusts its genetic operations to generate effective adversarial passages with significantly reduced time and memory usage. Our experimental evaluation shows that DIGA achieves superior efficiency and scalability compared to existing methods, while maintaining comparable or better attack success rates across multiple datasets.
Introduction
Retrieval-Augmented Generation (RAG) has become a cornerstone for deploying Large Language Models (LLMs) in production, effectively mitigating hallucinations by grounding model outputs in verified external data. However, this architectural dependency introduces a critical security vulnerability known as corpus poisoning. In these attacks, an adversary injects a single malicious document into the retrieval corpus. If this document is retrieved for a wide range of user queries, it can manipulate the LLM's output, spread misinformation, or facilitate indirect prompt injections.
The Challenge of Corpus Poisoning
Current state-of-the-art attacks face significant hurdles. White-box attacks, like HotFlip, require access to the retriever's internal gradients—a luxury rarely available with proprietary APIs like those from OpenAI or Cohere. Conversely, existing black-box attacks often suffer from extreme computational overhead or poor generalization across different datasets. For instance, methods like Vec2Text require training complex inversion models for every new target domain, limiting their practical scalability.
The DIGA Method: Exploiting Retriever Quirks
To overcome these limitations, we introduce the Dynamic Importance-Guided Genetic Algorithm (DIGA). DIGA is designed around two fundamental observations of how modern dense retrievers process information:
- Insensitivity to Order: Dense retrievers often behave like high-dimensional "bag-of-words" models. Scrambling the word order of a sentence typically results in a nearly identical embedding.
- Bias Toward Influential Tokens: Retrieval scores are disproportionately influenced by a small subset of "heavy-lifting" tokens (e.g., "fastest", "animal") rather than common filler words.
The DIGA Pipeline
DIGA utilizes a genetic algorithm optimized for these properties:
- Token Scoring via TF-IDF: Instead of expensive neural scoring, DIGA uses classical TF-IDF to identify influential tokens across the corpus. This allows the attack to remain entirely black-box and computationally light.
- Smart Mutation: During the evolutionary process, DIGA protects tokens with high importance scores from being mutated, ensuring the "influential" core of the adversarial passage is preserved.
- Hybrid Optimization: The process begins with DIGA to establish a strong "bag of words" foundation (80% of iterations) and concludes with a vanilla genetic algorithm to refine the final passage.
Experimental Evaluation
We evaluated DIGA across four diverse datasets from the BEIR benchmark: Natural Questions (NQ), NFCorpus (Medical), FiQA (Finance), and SciDocs (Scientific).
Key Results
- Unprecedented Efficiency: DIGA is 6.5x to 9.3x faster than the gradient-based HotFlip and requires 6x to 10.5x less GPU memory.
- Superior Black-Box Performance: DIGA consistently outperformed other black-box baselines like Vec2Text in Attack Success Rate (ASR).
- Cross-Retriever Transferability: Adversarial passages generated by DIGA often successfully trick sparse retrievers (like BM25) because the algorithm naturally converges on the same keywords used by traditional indexing systems.
- Indirect Prompt Injection: In practical tests, DIGA successfully injected malicious instructions (e.g., forcing the model to say "I don't know") into RAG pipelines, achieving a 67% success rate when the poisoned passage was retrieved.
Conclusion and Future Work
DIGA demonstrates that highly effective corpus poisoning attacks do not require internal model access or massive compute. By exploiting the inherent biases of dense retrievers toward specific tokens, DIGA provides a scalable and efficient tool for auditing RAG security. While a performance gap remains between black-box and white-box methods, DIGA's success highlights the need for more robust retrieval architectures that move beyond simple token-level influence. Future research should focus on developing defenses, such as perplexity-based filtering, to safeguard RAG systems against these evolving threats.
Summary updated