TL;DR
This paper reveals a major "alignment gap" in Retrieval-Augmented Generation (RAG) security: what researchers claim to protect often doesn't match what they actually measure. By auditing 118 RAG defense papers, the authors show that current "verbatim copying" guards are easily bypassed by adaptive attackers who can still extract nearly half of the underlying private facts even when word-for-word copying is reduced to zero.
What Problem Does This Paper Address?
As RAG systems become the standard for connecting LLMs to private or copyrighted data, a "protection" literature has exploded. However, the term "protection" has become a catch-all that obscures vastly different technical goals.
The authors argue that comparing these defenses is currently like "comparing a burglar alarm to a fireproof safe." One paper might claim to protect copyright by planting a watermark (detection), while another does it by blocking unauthorized users (access control). Without a shared language for measured endpoints—the exact technical event being tested—we cannot tell which defenses actually work. Furthermore, most evaluations rely on "toy" benchmarks (like Wikipedia) that don't contain the sensitive information these systems are meant to protect in the real world.
Methods
The researchers conducted a systematic survey and human-coded evidence map of 118 full-text RAG protection papers. To do this, they coordinated 32 annotators to categorize research based on a new seven-family taxonomy of measured endpoints.
Beyond the survey, they built a controlled RAG testbed to stress-test common defenses. They evaluated four representative controls (including access filtering and output guards) across several models (Qwen3-8B, Phi-4-14B, etc.) using a three-query adaptive attacker designed to iteratively extract information even when surface-level copying is blocked.
Datasets
The survey found that the field relies heavily on standard open-domain QA benchmarks:
- Natural Questions (35 records)
- HotpotQA (31 records)
- MS MARCO (28 records)
In contrast, datasets with genuinely sensitive content like the Enron Email corpus or clinical notes (ChatDoctor) are rarely used. For their own experimental testbed, the authors introduced a 192-document security corpus spanning Wikipedia and scientific papers, paired with human-authored "fact probes" to measure semantic leakage.
Key Insights and Ideas
- Mechanisms do not dictate outcomes: Knowing that a paper uses "context rewriting" tells you almost nothing about what it protects. The survey found that popular interventions like reranking and rewriting are used to target up to six different endpoint families.
- The "Copyright" Blind Spot: 73.3% of papers claiming to protect copyright actually only measure usage detection (watermarking). None of the surveyed copyright papers primarily measured whether the model actually stopped emitting copyrighted expressions.
- Surface Copying $\neq$ Fact Secrecy: This is the most critical "aha moment." A defense can be 100% successful at stopping a model from quoting a source verbatim while still being a total failure at keeping the underlying facts secret.
- The Failure of Static Testing: Most papers test against a single, fixed prompt. The authors' testbed showed that while a defense might look robust after one query, its "hidden-fact recall" can jump from ~3% to ~48% by the third turn of an adaptive conversation.
Key Results
- The Literature is Over-Optimistic: 92% of surveyed papers report that their intervention improves protection. However, the authors attribute this to "gentle" evaluation conditions: non-adaptive attackers and datasets with little sensitive info.
- The 8-Gram Guard Paradox: In the authors' testbed, adding a normalized 8-gram output guard (which redacts matching spans) successfully drove verbatim copying down to 0.0%.
- Adaptive Leakage: Despite that 0% copying score, a three-query adaptive attacker was still able to recover 48.3% of hidden source facts on Qwen3-8B.
- Metadata Fragility: Structural access controls (Family D) are only as good as their labels. A mere 12.5% rate of missing metadata labels in a "fail-open" policy restored verbatim copying to nearly baseline levels.
Limitations and Future Work
The authors acknowledge that their survey is limited by the reporting quality of the original papers; many studies fail to report uncertainty (confidence intervals) or provide the artifacts necessary for independent verification.
They propose Three Golden Rules for future RAG protection research:
- Measure content disclosure directly: If you claim to protect privacy, you must measure what the user actually receives, not just a proxy like "perplexity."
- Test against adaptive adversaries: Single-query evaluations are insufficient for conversational AI.
- Validate evidential sufficiency: Researchers must prove their attack is actually strong on an unprotected baseline before claiming their defense is the reason for a low leakage score.
Summary updated