Protecting Copyrighted Material with Unique Identifiers in Large Language Model Training

Shuai Zhao, Linchao Zhu, Ruijie Quan, Yi YangSubmission to ICLR 2025 · Withdrawn

TL;DR

This paper proposes an "insert-and-detect" framework that allows everyday web creators and content platforms to prove whether their copyrighted data was used to train Large Language Models (LLMs). By embedding "ghost sentences"—unique passphrases made of randomly combined natural words—into published content, users can verify model infringement via a simple conversational completion test (the last-$k$ words test) or an alternative perplexity test.

What Problem Does This Paper Address?

LLMs are routinely pre-trained on massive web crawls without transparent disclosure of training data sources, raising severe copyright infringement concerns. While researchers have proposed Membership Inference Attacks (MIAs) to detect whether specific data was in the training corpus, existing MIAs suffer from two critical flaws:

  1. High false-positive rates: Massively scaled training corpora contain many semantically similar texts, making it difficult to distinguish an exact training document from related web data.
  2. High technical barrier: Most MIAs rely on complex loss manipulation, shadow model training, or API-level probability access. General users lack the coding skills and infrastructure to run these audits, forcing reliance on centralized auditing services.

Methods

The authors introduce an insert-and-detect framework centered on ghost sentences:

  • Ghost Sentence Construction: Users generate unique passphrases using Diceware (e.g., drawing words randomly from the Electronic Frontier Foundation's 7,776-word list). Because combinations of random words lack semantic continuity, their occurrence in an untrained model has near-zero probability. Creators can prepend natural language prefixes (e.g., "Please ignore the following:") to evade simple heuristic filters.
  • Last-$k$ Words Test (Conversational MIA): For passphrases repeated sufficiently in training data ($\ge 10$ times), models memorize them verbatim. A user simply provides the preceding context and prompts the LLM to complete the final $k$ words ($k = 1$ or $2$). A one-proportion $z$-test shows that correctly completing even 1–2 random words yields statistical significance ($p < 0.01$), making it practically impossible for the model to guess by chance.
  • Perplexity Test: For cases where passphrases appear fewer times ($\mu \approx 5\text{--}7$), the model might not complete them verbatim but will still assign them abnormally low perplexity compared to completely unseen random passphrases. A hypothesis test against an empirical perplexity distribution enables reliable detection.

Datasets

The authors evaluated their methods across multiple datasets:

  • Webis-TLDR-17: A large Reddit post/comment corpus (~3.8M documents from 1.4M users). The authors created a 148K subset (Webis-148K) containing 8,192 users with 10–200 posts to simulate realistic multi-document authors.
  • LaMini-Instruction & OpenOrca: Combined with Webis data to scale pre-training experiments up to 9.8M examples (~3B tokens).
  • Domain Datasets: 100K medical dialogues from HealthCareMagic and 120K code samples from Code-Instructions-120k-Alpaca.
  • Downstream Benchmarks: HellaSwag and MMLU were used to verify that ghost sentences did not degrade general model capabilities.

Key Insights and Ideas

  • User-Level Aggregation Helps: Even if a single user's ghost sentence has relatively low repetition, increasing the total number of distinct ghost sentences embedded across different users in the corpus actually boosts individual memorization rates.
  • Placement Matters Greatly: Inserting ghost sentences in the latter half of a document yields dramatically higher memorization (over 90% document accuracy) compared to placing them in the middle (~3.6%), where the model heavily conditions on surrounding context.
  • Scaling and Alignment Invariance: Larger models (e.g., 7B vs. 1.1B) memorize ghost sentences at much lower repetition counts. Crucially, post-pretraining alignment via Direct Preference Optimization (DPO) does not erase the memorization of these identifiers.
  • No Linguistic Penalty: Inserting ghost sentences into training corpora has virtually no negative impact on standard benchmark scores (e.g., MMLU and HellaSwag remain steady).

Key Results

  • Chat-Based Identification: In an OpenLLaMA-3B model trained on 1.8M documents where ghost sentences accounted for only 0.0017% of all tokens, 11 out of 16 users (68.75%) successfully verified membership using the last-1 word test (with an average repetition $\mu = 24$).
  • Perplexity Test under Low Repetition: On fine-tuned LLaMA-13B, testing 30 ghost sentences with an average repetition of only 7 reached an AUC of 0.891 (and 0.991 AUC at $\mu = 9$).
  • Code Memorization: Ghost sentences embedded in code repositories achieved exceptional detection rates (99.22% User-Accuracy and 98.44% Document-Accuracy at $k=2$), making the method especially well-suited for open-source codebases.

Limitations and Future Work

  • Susceptibility to Targeted Filtering: Because ghost sentences follow recognizable syntactic anomalies, aggressive model developers could train classifiers to scrub Diceware-style sequences prior to training.
  • Deduplication Constraints: Long passphrases (>50 tokens) risk removal by exact substring deduplication pipelines. The authors recommend keeping passphrases around 10 words (~22 BPE tokens).
  • Repetition Thresholds: Single-document creators cannot trigger verbatim memorization on their own and would need platform-level coordination (e.g., Reddit or Medium injecting platform-wide identifiers across multiple posts).
  • Future Directions: Developing more linguistically natural, diverse identifier schemes that are resistant to automated data-cleaning pipelines.

Summary updated

Abstract

A primary concern regarding training large language models (LLMs) is whether they abuse copyrighted online text. With the increasing training data scale and the prevalence of LLMs in daily lives, two problems arise: (1) false positive membership inference results misled by similar examples; (2) membership inference methods are usually too complex for general users to understand and use. To address these issues, we propose an alternative \textit{insert-and-detect} methodology, advocating that web users and content platforms employ \textbf{\textit{unique identifiers}} for reliable and independent membership inference. Users and platforms can create their identifiers, embed them in copyrighted text, and independently detect them in future LLMs. As an initial demonstration, we introduce \textit{\textbf{ghost sentences}} and a user-friendly last-$k$ words test, allowing general users to chat with LLMs for membership inference. Ghost sentences consist primarily of unique passphrases of random natural words, which can come with customized elements to bypass possible filter rules. The last-$k$ words test requires a significant repetition time of ghost sentences~($\ge10$). For cases with fewer repetitions, we designed an extra perplexity test, as LLMs exhibit high perplexity when encountering unnatural passphrases. We also conduct a comprehensive study on the memorization and membership inference of ghost sentences, examining factors such as training data scales, model sizes, repetition times, insertion positions, wordlist of passphrases, alignment, \textit{etc}. Our study shows the possibility of applying ghost sentences in real scenarios and providing instructions for the potential application.
BibTeX
@misc{zhao2024protecting,
  author = {Shuai Zhao and Linchao Zhu and Ruijie Quan and Yi Yang},
  title = {{Protecting Copyrighted Material with Unique Identifiers in Large Language Model Training}},
  year = {2024},
  eprint = {2403.15740},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2403.15740},
}

Loading related content...

Figure