Protecting Copyrighted Material with Unique Identifiers in Large Language Model Training

Shuai Zhao, Linchao Zhu, Ruijie Quan, Yi Yang

TL;DR

This paper introduces "ghost sentences"—unique, random passphrases embedded into online content—as a "canary in the coal mine" for copyright protection. By using these identifiers, creators can catch Large Language Models (LLMs) "red-handed" if they are trained on copyrighted data, using either a simple chat-based completion test or a more technical perplexity analysis.

What Problem Does This Paper Address?

As LLMs are trained on increasingly massive scrapes of the open internet, creators (from poets to programmers) are rightfully concerned that their copyrighted work is being used without permission.

Current methods to prove a model was trained on specific data—known as Membership Inference Attacks (MIA)—suffer from two major flaws:

  1. False Positives: If a model is trained on a text similar to yours, an MIA might incorrectly flag your text as part of the training set.
  2. Complexity: Most detection methods require deep expertise in statistics or coding, making them inaccessible to the average web user.

The authors propose an "insert-and-detect" methodology that allows users to independently and reliably verify if their data was ingested.

Methods

The core of the proposal is the Ghost Sentence: a string of random, natural words generated using a "diceware" method (e.g., cupbearer tinsmith richly automatic rewash liftoff...). Because these sequences are statistically improbable in natural language, they serve as unique, unmistakable identifiers.

The authors propose two ways to detect these ghosts:

  • The Last-k Words Test: A user-friendly method where a person simply asks the LLM to complete the end of a ghost sentence. If the model can finish the random passphrase verbatim, it is statistically certain that the model encountered that specific sentence during training.
  • The Perplexity Test: For cases where the data was only repeated a few times, the authors use "perplexity"—a measure of how "surprised" a model is by a sequence of words. Since random word strings are normally very confusing to an LLM, a significantly low perplexity score indicates the model has seen that specific string before.

Datasets

The researchers evaluated their methodology across several contexts:

  • Webis-TLDR-17: A massive dataset of 3.8 million Reddit posts used to simulate real-world web user behavior.
  • Webis-148K: A curated subset used for instruction-tuning experiments.
  • HealthCareMagic & Code Datasets: Used to test how ghost sentences perform in specialized domains like medical conversations and programming.
  • LaMini-Instruction & OpenOrca: Used for broader pre-training and continual training evaluations.

Key Insights and Ideas

  • The Power of Repetition: For a model to "memorize" a ghost sentence well enough for a chat-based test, the sentence needs to appear at least 10 times in the training data. However, the perplexity test can detect "ghosts" with as few as 5–7 repetitions.
  • Positioning is Key: Ghost sentences are more likely to be memorized if they are placed in the latter half of a document. This is likely because LLMs are trained to predict the next token based on previous context; a random string at the end of a coherent story stands out more.
  • Model Size Matters: Larger models (like LLaMA-13B) are much better at memorizing these identifiers than smaller models (like TinyLlama-1.1B), meaning they require fewer repetitions to be caught.
  • Alignment Doesn't Hide the Evidence: Safety training and alignment (like RLHF or DPO) do not significantly affect the model's memorization of these unique identifiers.

Key Results

  • High Reliability: In experiments with OpenLLaMA-3B, 11 out of 16 users successfully identified their data using the chat-based test.
  • Low Footprint: Ghost sentences are highly efficient; they can be detected even when they account for as little as 0.0017% of the total training tokens.
  • Statistical Significance: For a LLaMA-13B model, the perplexity test achieved an ROC AUC of 0.891 for ghost sentences repeated only 7 times, providing a very strong signal of data inclusion.
  • Proof of Concept: As a final flourish, the authors embedded a 12-word unique identifier for the first author at the very end of the paper (quickstep drudge consent wackiness...), daring future LLMs to memorize it.

Limitations and Future Work

  • The "Arms Race": If ghost sentences become common, model providers might develop filters to strip out random word sequences during data cleaning. The authors suggest that users can bypass this by adding "customized elements" or prefixes to make the sentences look more like natural text.
  • Deduplication: Very long ghost sentences might be caught by standard "deduplication" algorithms used by AI companies to clean data. The authors recommend keeping passphrases around 10 words (roughly 22 tokens) to avoid this.
  • Future Directions: The authors hope this work sparks a movement toward more diverse, user-friendly unique identifiers that can work together to make data filtering "intractable" for those scraping the web without permission.

Summary updated

Abstract

A primary concern regarding training large language models (LLMs) is whether they abuse copyrighted online text. With the increasing training data scale and the prevalence of LLMs in daily lives, two problems arise: \textbf{1)} false positive membership inference results misled by similar examples; \textbf{2)} membership inference methods are usually too complex for end users to understand and use. To address these issues, we propose an alternative \textit{insert-and-detect} methodology, advocating that web users and content platforms employ \textbf{\textit{unique identifiers}} for reliable and independent membership inference. Users and platforms can create their identifiers, embed them in copyrighted text, and independently detect them in future LLMs. As an initial demonstration, we introduce \textit{\textbf{ghost sentences}} and a user-friendly last-$k$ words test, allowing end users to chat with LLMs for membership inference. Ghost sentences consist primarily of unique passphrases of random natural words, which can come with customized elements to bypass possible filter rules. The last-$k$ words test requires a significant repetition time of ghost sentences~($\ge10$). For cases with fewer repetitions, we designed an extra perplexity test, as LLMs exhibit high perplexity when encountering unnatural passphrases. We also conduct a comprehensive study on the memorization and membership inference of ghost sentences, examining factors such as training data scales, model sizes, repetition times, insertion positions, wordlist of passphrases, alignment, \textit{etc}. Our study shows the possibility of applying ghost sentences in real scenarios and provides instructions for the potential application.
BibTeX
@misc{zhao2024protecting,
  author = {Shuai Zhao and Linchao Zhu and Ruijie Quan and Yi Yang},
  title = {{Protecting Copyrighted Material with Unique Identifiers in Large Language Model Training}},
  year = {2024},
  eprint = {2403.15740},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2403.15740},
}
Figure