EcoSafeRAG: Efficient Security through Context Analysis in Retrieval-Augmented Generation

Ruobing Yao, Yifei Zhang, Shuang Song, Neng Gao, Chenyang TuEMNLP 2025

TL;DR

This paper introduces EcoSafeRAG, a novel defense mechanism for Retrieval-Augmented Generation (RAG) systems that enhances security against malicious attacks without relying on the LLM's internal knowledge. By employing sentence-level segmentation and a unique bait-guided contextual diversity check, EcoSafeRAG effectively filters out poisoned content, improving both security and performance in clean scenarios while maintaining practical operational costs.

What Problem Does This Paper Address?

Retrieval-Augmented Generation (RAG) has become a powerful technique to imbue Large Language Models (LLMs) with up-to-date and external knowledge, leading to more accurate and contextually relevant responses. However, this reliance on external knowledge bases introduces new vulnerabilities, particularly "corpus poisoning" attacks. In these attacks, malicious documents are subtly injected into the knowledge base, designed to trick the RAG system into generating harmful or incorrect outputs. Existing defense mechanisms often rely on the LLM's internal knowledge, which contradicts the core RAG principle of leveraging external information and can introduce model biases. This paper addresses the critical need for a RAG defense system that is secure, efficient, and independent of the LLM's internal knowledge.

Methods

EcoSafeRAG introduces a multi-stage defense approach:

  • Sentence-level Segmentation: Instead of processing entire documents, EcoSafeRAG breaks down retrieved passages into individual sentences. This strategy helps expose attack features more clearly and reduces redundancy in normal content.
  • Core Sentence Screening: Candidate sentences are screened based on their semantic similarity to the original query, using both an adaptive threshold (based on the maximum similarity) and an absolute threshold to ensure high relevance.
  • Bait-guided Contextual Diversity Check: This is the core innovation. It leverages the observation that poisoned documents often exhibit unnatural contextual patterns. By introducing "bait" sentences (crafted to mimic suspicious patterns), EcoSafeRAG can identify clusters of potentially malicious content. This method leverages systematic differences in contextual quality between normal and poisoned documents, enabling precise filtration without relying on LLM internal knowledge.
  • LLM Generation: Finally, the LLM generates a response using only the filtered, clean sentences, respecting a token budget.

Datasets

The authors evaluated EcoSafeRAG on three standard open-domain question-answering datasets:

  • Natural Questions (NQ)
  • HotpotQA
  • MS-MARCO

They used attack-augmented questions from these datasets, pairing each question with up to 5 crafted adversarial documents.

Key Insights and Ideas

  • Independence from LLM Internal Knowledge: A major breakthrough is the development of a defense that doesn't require the LLM's internal knowledge, aligning with RAG's design principles and avoiding model bias.
  • Bait-guided Diversity as a Novel Defense: The bait-guided contextual diversity check is a clever way to exploit the structural differences between legitimate and poisoned content, especially effective in cold-start scenarios (few poisoned examples).
  • Sentence-level Segmentation Enhances Detection: Breaking down documents into sentences makes it easier to spot subtle anomalies and improves the overall signal-to-noise ratio for defense mechanisms.
  • Trade-off Between Security and Performance: EcoSafeRAG demonstrates that it's possible to achieve high security without significantly sacrificing performance in clean scenarios, and even improving it in some cases.

Key Results

  • State-of-the-Art Security: EcoSafeRAG achieved the lowest Attack Success Rate (ASR) across all tested attack scenarios (GCG, PIA, Poison) and language models, with a maximum ASR of only 3%.
  • Improved Clean Scenario Performance: EcoSafeRAG not only defends against attacks but also improves accuracy in clean scenarios, showing an average increase of 17% on Vicuna.
  • Efficiency: The system operates with approximately 1.2x the latency of vanilla RAG but reduces input token requirements by 40%-80%, making it computationally efficient.
  • Robustness: The defense remains effective even with varying numbers of poisoned documents (N=1 to N=5) and across different model architectures.

Limitations and Future Work

The authors acknowledge a few limitations and suggest future directions:

  • Dataset-Specific Tuning: The optimal DBSCAN epsilon parameter currently requires dataset-specific tuning, and automating this process is a potential area for future work.
  • Evolving Attack Patterns: The bait design relies on knowledge of existing attack patterns, necessitating continuous updates as attack strategies evolve.
  • Computational Overhead of Segmentation: While sentence segmentation is beneficial, it can introduce computational overhead in large-scale deployments. Pre-computed indexing strategies could mitigate this.

Summary updated

BibTeX
@inproceedings{yao2025ecosaferag,
  author = {Ruobing Yao and Yifei Zhang and Shuang Song and Neng Gao and Chenyang Tu},
  title = {{EcoSafeRAG: Efficient Security through Context Analysis in Retrieval-Augmented Generation}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2025},
}
Figure