RAG-Pull: Imperceptible Attacks on RAG Systems for Code Generation

Vasilije Stambolic, Aritra Dhar, Lukas Cavigelli

Abstract

Retrieval-Augmented Generation (RAG) increases the reliability and trustworthiness of the LLM response and reduces hallucination by eliminating the need for model retraining. It does so by adding external data into the LLM's context. We develop a new class of black-box attack, RAG-Pull, that inserts hidden UTF characters into queries or external code repositories, redirecting retrieval toward malicious code, thereby breaking the models' safety alignment. We observe that query and code perturbations alone can shift retrieval toward attacker-controlled snippets, while combined query-and-target perturbations achieve near-perfect success. Once retrieved, these snippets introduce exploitable vulnerabilities such as remote code execution and SQL injection. RAG-Pull's minimal perturbations can alter the model's safety alignment and increase preference towards unsafe code, therefore opening up a new class of attacks on LLMs.

Introduction

Retrieval-Augmented Generation (RAG) has become the gold standard for enhancing Large Language Models (LLMs) with up-to-date, domain-specific knowledge. By retrieving relevant snippets from external repositories, RAG systems significantly reduce hallucinations and keep model knowledge current. However, this reliance on a retriever introduces a novel attack surface. While traditional prompt injection or data poisoning attacks often rely on visible, suspicious text, RAG-Pull introduces a stealthy, black-box attack using invisible Unicode characters. This method tricks RAG systems into serving malicious code without alerting human developers, effectively "pulling" the LLM away from its safety alignment.

Methodology

RAG-Pull exploits the process of converting text into high-dimensional vector embeddings. The attack mechanism consists of three primary components:

  1. Invisible Perturbations: The attack utilizes 382 specific Unicode characters (e.g., zero-width spaces, variation selectors) that are functionally present in the text but do not render visually in most code editors or browsers.
  2. Differential Evolution (DE): As a black-box attack, RAG-Pull employs DE, a gradient-free optimization algorithm. DE iteratively optimizes the placement of invisible characters to minimize the mathematical distance between the query embedding and a malicious target embedding.
  3. Attack Scenarios: The researchers identified three primary vectors:
  4. Query Perturbation: Manipulating the user's query through intermediary platforms (e.g., a prompt engineering website).
  5. Target Perturbation: Poisoning external repositories with "infected" malicious snippets.
  6. Combined Attack: Simultaneously perturbing both the query and the target for maximum efficacy.
Figure 1: Overview of the RAG-Pull attack architecture, showing how engineered prompts and poisoned external data sources can redirect LLM responses.

Experimental Evaluation

The effectiveness of RAG-Pull was evaluated using Java (Vulnerability Fix Dataset) and Python (Python Alpaca, CyberNative Security DPO) datasets. Malicious payloads included Remote Code Execution (RCE), SQL injection, and command injection.

The results demonstrate the extreme fragility of embedding spaces. In the combined attack scenario, RAG-Pull achieved a 100% retrieval success rate across nearly all test cases. Furthermore, the generation model (Codestral-22B) incorporated the retrieved malicious code into its final output with up to 99.44% success.

Figure 4: t-SNE visualization of embedding shifts. As the percentage of perturbations increases from 0% to 50%, the query embedding (blue) is pulled significantly closer to the malicious target embedding (red).

Key Insights

  • The Fragility of Embeddings: High-dimensional vectors can be steered with minimal, human-invisible changes. Inserting just a few invisible characters can drastically change a snippet's location in the embedding space without changing its semantic meaning to a human.
  • Query Power vs. Target Power: Perturbing the query is significantly more effective than perturbing the target alone. Query perturbations disrupt the semantic structure the retriever is looking for, making it easier to hijack the search.
  • Exploiting Trust: LLMs are often prompted to treat retrieved RAG context as "ground truth." RAG-Pull leverages this trust to override the LLM's internal safety training, causing it to suggest vulnerable code that a developer might copy-paste without ever seeing the hidden triggers.

Conclusion and Future Work

RAG-Pull represents a significant threat to the security of RAG-enabled code generation. While sanitizing Unicode characters is a potential defense, it risks breaking legitimate functionality (e.g., emojis or non-Latin scripts). Future research is needed to develop more robust, "Unicode-aware" embedding models and security layers that can distinguish between benign and adversarial invisible characters in automated pipelines.

Summary updated

Abstract

Retrieval-Augmented Generation (RAG) increases the reliability and trustworthiness of the LLM response and reduces hallucination by eliminating the need for model retraining. It does so by adding external data into the LLM's context. We develop a new class of black-box attack, RAG-Pull, that inserts hidden UTF characters into queries or external code repositories, redirecting retrieval toward malicious code, thereby breaking the models' safety alignment. We observe that query and code perturbations alone can shift retrieval toward attacker-controlled snippets, while combined query-and-target perturbations achieve near-perfect success. Once retrieved, these snippets introduce exploitable vulnerabilities such as remote code execution and SQL injection. RAG-Pull's minimal perturbations can alter the model's safety alignment and increase preference towards unsafe code, therefore opening up a new class of attacks on LLMs.
BibTeX
@misc{stambolic2025rag,
  author = {Vasilije Stambolic and Aritra Dhar and Lukas Cavigelli},
  title = {{RAG-Pull: Imperceptible Attacks on RAG Systems for Code Generation}},
  year = {2025},
  eprint = {2510.11195},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2510.11195},
}

Loading related content...

Figure