TL;DR
This paper proposes an "insert-and-detect" framework that allows everyday web creators and content platforms to prove whether their copyrighted data was used to train Large Language Models (LLMs). By embedding "ghost sentences"—unique passphrases made of randomly combined natural words—into published content, users can verify model infringement via a simple conversational completion test (the last-$k$ words test) or an alternative perplexity test.
What Problem Does This Paper Address?
LLMs are routinely pre-trained on massive web crawls without transparent disclosure of training data sources, raising severe copyright infringement concerns. While researchers have proposed Membership Inference Attacks (MIAs) to detect whether specific data was in the training corpus, existing MIAs suffer from two critical flaws:
- High false-positive rates: Massively scaled training corpora contain many semantically similar texts, making it difficult to distinguish an exact training document from related web data.
- High technical barrier: Most MIAs rely on complex loss manipulation, shadow model training, or API-level probability access. General users lack the coding skills and infrastructure to run these audits, forcing reliance on centralized auditing services.
Methods
The authors introduce an insert-and-detect framework centered on ghost sentences:
- Ghost Sentence Construction: Users generate unique passphrases using Diceware (e.g., drawing words randomly from the Electronic Frontier Foundation's 7,776-word list). Because combinations of random words lack semantic continuity, their occurrence in an untrained model has near-zero probability. Creators can prepend natural language prefixes (e.g., "Please ignore the following:") to evade simple heuristic filters.
- Last-$k$ Words Test (Conversational MIA): For passphrases repeated sufficiently in training data ($\ge 10$ times), models memorize them verbatim. A user simply provides the preceding context and prompts the LLM to complete the final $k$ words ($k = 1$ or $2$). A one-proportion $z$-test shows that correctly completing even 1–2 random words yields statistical significance ($p < 0.01$), making it practically impossible for the model to guess by chance.
- Perplexity Test: For cases where passphrases appear fewer times ($\mu \approx 5\text{--}7$), the model might not complete them verbatim but will still assign them abnormally low perplexity compared to completely unseen random passphrases. A hypothesis test against an empirical perplexity distribution enables reliable detection.
Datasets
The authors evaluated their methods across multiple datasets:
- Webis-TLDR-17: A large Reddit post/comment corpus (~3.8M documents from 1.4M users). The authors created a 148K subset (Webis-148K) containing 8,192 users with 10–200 posts to simulate realistic multi-document authors.
- LaMini-Instruction & OpenOrca: Combined with Webis data to scale pre-training experiments up to 9.8M examples (~3B tokens).
- Domain Datasets: 100K medical dialogues from HealthCareMagic and 120K code samples from Code-Instructions-120k-Alpaca.
- Downstream Benchmarks: HellaSwag and MMLU were used to verify that ghost sentences did not degrade general model capabilities.
Key Insights and Ideas
- User-Level Aggregation Helps: Even if a single user's ghost sentence has relatively low repetition, increasing the total number of distinct ghost sentences embedded across different users in the corpus actually boosts individual memorization rates.
- Placement Matters Greatly: Inserting ghost sentences in the latter half of a document yields dramatically higher memorization (over 90% document accuracy) compared to placing them in the middle (~3.6%), where the model heavily conditions on surrounding context.
- Scaling and Alignment Invariance: Larger models (e.g., 7B vs. 1.1B) memorize ghost sentences at much lower repetition counts. Crucially, post-pretraining alignment via Direct Preference Optimization (DPO) does not erase the memorization of these identifiers.
- No Linguistic Penalty: Inserting ghost sentences into training corpora has virtually no negative impact on standard benchmark scores (e.g., MMLU and HellaSwag remain steady).
Key Results
- Chat-Based Identification: In an OpenLLaMA-3B model trained on 1.8M documents where ghost sentences accounted for only 0.0017% of all tokens, 11 out of 16 users (68.75%) successfully verified membership using the last-1 word test (with an average repetition $\mu = 24$).
- Perplexity Test under Low Repetition: On fine-tuned LLaMA-13B, testing 30 ghost sentences with an average repetition of only 7 reached an AUC of 0.891 (and 0.991 AUC at $\mu = 9$).
- Code Memorization: Ghost sentences embedded in code repositories achieved exceptional detection rates (99.22% User-Accuracy and 98.44% Document-Accuracy at $k=2$), making the method especially well-suited for open-source codebases.
Limitations and Future Work
- Susceptibility to Targeted Filtering: Because ghost sentences follow recognizable syntactic anomalies, aggressive model developers could train classifiers to scrub Diceware-style sequences prior to training.
- Deduplication Constraints: Long passphrases (>50 tokens) risk removal by exact substring deduplication pipelines. The authors recommend keeping passphrases around 10 words (~22 BPE tokens).
- Repetition Thresholds: Single-document creators cannot trigger verbatim memorization on their own and would need platform-level coordination (e.g., Reddit or Medium injecting platform-wide identifiers across multiple posts).
- Future Directions: Developing more linguistically natural, diverse identifier schemes that are resistant to automated data-cleaning pipelines.
Summary updated