TL;DR
Standard metrics like Attack Success Rate (ASR) are misleading for evaluating RAG defenses because they hide "introduced" failures and side effects. This paper introduces Aligned RAG Evaluation, a matched-query framework that reveals how defenses can inadvertently introduce new vulnerabilities while attempting to block others, and why current poisoning detectors fail when the "poison" budget increases.
What Problem Does This Paper Address?
Retrieval-Augmented Generation (RAG) systems are highly effective but vulnerable to knowledge poisoning: an attacker injects a malicious document into the retrieval corpus designed to steer the model toward a specific, incorrect answer.
Currently, researchers evaluate defenses using a single aggregate number: the Attack Success Rate (ASR). The authors argue this is a "blind spot." A defense might lower the ASR by blocking some attacks, but it might also:
- Introduce new failures: Cause the model to output the attacker's target on queries that were previously safe.
- Degrade clean performance: Break the model’s ability to answer legitimate questions correctly.
- Create "false positives" in evaluation: Simple string matching often counts a model's rejection of a poisoned passage (e.g., "The answer is NOT X") as a successful attack just because the target string "X" appeared in the output.
Methods
The authors propose Aligned RAG Evaluation (ALIGNEDRAG-EVAL), which moves beyond aggregate scores to a matched-query protocol.
The Four-Condition Matrix
For every query, the framework saves four distinct answers to compare behavior across different states:
- Clean retrieval, No defense ($T_{c0}$): The baseline performance.
- Clean retrieval, With defense ($T_{c1}$): Measures the "utility cost" or side effects of the defense.
- Poisoned retrieval, No defense ($T_{p0}$): Measures the raw vulnerability to the attack.
- Poisoned retrieval, With defense ($T_{p1}$): Measures the actual effectiveness of the defense.
New Metrics
Instead of just ASR, they calculate:
- Net Reduction ($N$): Prevented attacks minus newly introduced attacks.
- Clean-target change ($H$): How often the defense introduces the target answer even when no poison is present.
- LLM-as-a-Judge: Using models like GPT-4o-mini to distinguish between a model mentioning a target and actually endorsing it as the correct answer.
Datasets
The evaluation uses two standard open-domain QA benchmarks:
- HotpotQA: A multi-hop reasoning dataset.
- Natural Questions (NQ): A dataset based on real Google search queries. Both are tested using the PoisonedRAG attack framework across different "poison budgets" (number of injected passages).
Key Insights and Ideas
- "Prevented" is not "Protected": A defense might stop 20 attacks but introduce 20 new ones on different queries. In aggregate ASR, this looks like zero progress, but the underlying system behavior has shifted significantly.
- The Endorsement Gap: String matching is a noisy proxy. The authors found that defenses often mention the attacker's target while explicitly rejecting it. String-based ASR counts these as failures, whereas an LLM judge correctly identifies them as successful defenses.
- Detector Collapse: Many defenses rely on detecting "outliers" in retrieval scores. However, if an attacker injects multiple passages (e.g., 5 instead of 1), these passages dominate the retrieved context. The "outlier" becomes the "norm," and the detector's effectiveness drops to near zero.
- Single-Passage Sufficiency: Most current poisoning attacks don't require complex reasoning across multiple documents. A single malicious passage is usually enough to flip the model's output, even if legitimate evidence is also present.
Key Results
- Introduced Failures: On HotpotQA ($p=1$), the TrustRAG defense prevented 20 target-positive answers but introduced 20 new ones, resulting in a net reduction of zero.
- Scoring Sensitivity: On NQ, TrustRAG appeared less effective than KeywordAgg under string matching (46 vs 42 positives). However, the LLM judge revealed the opposite: TrustRAG had far fewer endorsed targets (7 vs 20).
- The $p=5$ Cliff: Detectors that were nearly perfect at intercepting attacks with one poisoned passage (59/60) failed completely when five poisoned passages were retrieved (0/96), demonstrating a critical lack of robustness to higher attack budgets.
Limitations and Future Work
- Scale: The study used 100 queries per condition; larger-scale evaluations would provide tighter confidence intervals.
- Utility Metrics: While the paper focuses on security, the authors note that measuring "utility" (correctness on clean queries) remains difficult because exact-match metrics are too brittle for the long-form explanations often generated by defended systems.
- LLM Judge Uncertainty: The LLM judge sometimes returns "uncertain" labels, which complicates automated ranking.
- Future Directions: The authors suggest that future RAG defenses must move beyond simple outlier detection and focus on more robust evidence-composition analysis.
Summary updated