Abstract
Retrieval-Augmented Generation (RAG) systems are increasingly used to ground Large Language Models (LLMs) in external knowledge. However, this reliance introduces a significant security risk: knowledge corruption attacks, where poisoned data leads to hallucinations or malicious outputs. We introduce RAGDEFENDER, a lightweight, post-retrieval defense mechanism that identifies and filters adversarial passages before they reach the generator. By leveraging traditional machine learning techniques like clustering and semantic concentration analysis, RAGDEFENDER achieves near-zero attack success rates with minimal computational overhead and no additional GPU requirements.
Introduction
Retrieval-Augmented Generation (RAG) has become the industry standard for providing LLMs with up-to-date, domain-specific information. By retrieving relevant passages from an external database, RAG systems mitigate the inherent limitations of static model weights. However, this dependency creates a "knowledge corruption" vulnerability. Attackers can poison public or corporate knowledge bases with misleading information, which retrievers then inadvertently provide to the LLM.
Current defense strategies are often impractical for high-throughput production environments. Some require the LLM to self-reflect on every retrieved passage, which is prohibitively slow and expensive. Others necessitate fine-tuning specialized discriminator models, incurring high memory and maintenance costs. RAGDEFENDER addresses these challenges by providing a resource-efficient, LLM-agnostic defense layer.
Methodology: The RAGDEFENDER Pipeline
RAGDEFENDER operates as a two-stage filtering process situated between the retriever and the generator. Unlike previous methods that rely on heavy neural inference, RAGDEFENDER utilizes lightweight algorithms that run primarily on the CPU.
Stage 1: Grouping and Threat Estimation
The system first estimates the number of adversarial passages ($N_{adv}$) in the retrieved set. It employs two distinct strategies based on the query type:
- Clustering-based (Single-hop QA): This approach uses hierarchical agglomerative clustering and TF-IDF analysis. It exploits the fact that attackers often inject multiple redundant passages to ensure their "fake news" is selected by the retriever, creating dense semantic clusters.
- Concentration-based (Multi-hop QA): For complex queries requiring diverse facts, legitimate passages are typically semantically distinct. RAGDEFENDER identifies "concentration factors"—passages that are unusually similar to many others—to flag potential injections.
Stage 2: Identification and Filtering
After estimating $N_{adv}$, the system ranks passages by a "frequency score," which measures how often a passage appears in the most similar pairs within the retrieved set.
The top $N_{adv}$ passages are discarded, ensuring that only "safe," unpoisoned information is included in the final prompt sent to the LLM generator.
Experimental Evaluation
The authors evaluated RAGDEFENDER against three major benchmarks: Natural Questions (NQ), HotpotQA (multi-hop reasoning), and MS MARCO. The system was tested against sophisticated attack vectors, including PoisonedRAG, GARAG (typo-based evasion), and LLM-generated misleading content.
Key Results
- Effectiveness: On the Gemini model using the NQ dataset, RAGDEFENDER reduced the Attack Success Rate (ASR) from 0.89 to 0.02. This significantly outperformed the previous state-of-the-art, RobustRAG, which only achieved an ASR of 0.69 under similar conditions.
- Efficiency: RAGDEFENDER is approximately 12.36x faster than RobustRAG. While RobustRAG required over 23 seconds per iteration due to multiple LLM calls, RAGDEFENDER averaged just 0.77 seconds.
- Resource Usage: The defense requires 0 MB of additional GPU memory, making it highly suitable for production environments where GPU resources are at a premium.
Discussion and Limitations
While RAGDEFENDER provides a robust defense, certain limitations remain. The system's performance becomes less consistent when the retriever returns more than 10 passages ($k > 10$), which may impact "long-context" RAG applications. Additionally, while empirical results are strong, formal theoretical guarantees for the clustering behavior of adversarial passages are still being explored.
Future work will focus on extending RAGDEFENDER to multimodal (image/text) RAG systems and multilingual datasets. Furthermore, as attackers develop more "stealthy" poisoning strategies to avoid creating semantic clusters, the defense mechanism will need to evolve to counter these adaptive threats.
Conclusion
RAGDEFENDER demonstrates that effective security for RAG systems does not require massive computational resources. By turning the attacker's need for redundancy into a detection signal, RAGDEFENDER provides a fast, efficient, and LLM-agnostic shield against knowledge corruption, ensuring that RAG systems remain a reliable source of truth.
Summary updated