TL;DR
This paper introduces the use of lightweight, open-weight "decision models" (like Kev-4B) as security gates in Retrieval-Augmented Generation (RAG) pipelines. By screening retrieved HTML passages for hidden instructions and synthetic "pollution" before they reach the generator, these gates slash prompt injection success rates from 28% to under 6% at a fraction of the cost and latency of using a large LLM as a judge.
What Problem Does This Paper Address?
Retrieval-Augmented Generation (RAG) systems are increasingly used to ground LLMs in real-world data. However, this creates a massive security surface: the system "reads" text its operators didn't write. Attackers can exploit this via:
- Indirect Prompt Injection: Hiding instructions in web pages (e.g., in invisible HTML attributes like
aria-labeloralttext) that command the LLM to ignore its system prompt, leak data, or promote products. - Data Pollution: Flooding the web with synthetic, AI-generated "clickbait" or filler designed to outrank high-quality sources and mislead the generator.
While standard mitigations like HTML sanitization and citation prompting help, they aren't foolproof. Using a massive model like Llama-3-70B to "judge" every retrieved passage is effective but prohibitively expensive and slow for production systems.
Methods
The authors propose placing a Decision Model—a specialized, non-generative model that outputs scores for fixed questions—between the retriever and the generator.
Key aspects of their approach include:
- One-Pass Judgments: Unlike generative LLMs, decision models like Kev-4B can answer multiple questions (e.g., "Does this instruct an AI?", "Is this polluted?") in a single forward pass without generating tokens.
- HTML-Awareness: The gate reads raw HTML markup rather than just cleaned plain text. This is critical because many injections hide in attributes that sanitizers might miss but generators still "see."
- Tiered Policy:
- High Score: The passage is dropped entirely.
- Medium Score: The passage is kept, but the generator is forced into a "citation-only" mode to reduce hallucinations/obedience to injected text.
- Low Score: The passage passes through normally.
Datasets
The study utilizes two primary benchmarks:
- Corpus A (Injection): 1,000 queries based on Reddit threads. The authors built a custom generator to inject payloads into nine different HTML "carriers" (e.g., off-screen CSS, SVG titles, zero-width characters).
- Corpus B (Pollution): 200 MS MARCO queries mixed with synthetic clickbait generated by Mistral Small. In some tests, up to two-thirds of the candidate pool was synthetic.
Key Insights and Ideas
- HTML is the Better Signal: Gates perform significantly better when reading raw HTML (AUROC 0.94) compared to plain text (AUROC 0.87). Attackers love hiding instructions in metadata; if your filter only sees the "visible" text, it's already lost.
- Classification > Generation: The authors argue that "recognizing an instruction is a classification task." Small models trained specifically for typed judgments are more efficient and less prone to being "tricked" by the very instructions they are supposed to detect than generative models are.
- Placement Matters for Pollution: To fight data pollution, you cannot just filter the "top 10" results. If the pool is heavily polluted, filtering the top 10 might leave the generator with zero usable documents. The gate must screen a larger candidate pool before final ranking to ensure high-quality results "refill" the top slots.
Key Results
- Security: In the injection testbed, the Attack Success Rate (ASR) dropped from 27.9% (no defense) to 5.7% when the Kev-4B gate was added on top of standard mitigations.
- Efficiency: Kev-4B processed passages in just 76ms, costing roughly $0.015 per 1,000 passages. In contrast, a 70B LLM judge was 10x more expensive and significantly slower.
- Resilience to Obfuscation: The gate proved effective against character-level obfuscation (like using homoglyphs or zero-width insertions) that often bypasses simple keyword filters.
As shown in Figure 2, while standard mitigations (orange) fail against visible text and code blocks, the Kev gate (green) consistently suppresses attacks across almost all carriers.
Limitations and Future Work
- Hard Negatives: The "hard negative" samples (benign pages that look like instructions, such as "Click here to subscribe") were generated from templates. Real-world web text is much messier and might trigger higher false-positive rates.
- Adaptive Attackers: While the authors tested an attacker that could see the gate's scores, they did not test "white-box" attacks where the attacker has full access to the gate's weights to find adversarial examples.
- Scope: The study focused on text-based injections. Attacks via JavaScript or non-text media (images/audio) remain out of scope but are growing threats.
Summary updated