Ghost Sentence: A Tool for Everyday Users to Copyright Data from Large Language Models

Shuai Zhao, Linchao Zhu, Ruijie Quan, Yi Yang

Abstract

A primary concern regarding training large language models (LLMs) is whether they abuse copyrighted online text. With the increasing training data scale and the prevalence of LLMs in daily lives, two problems arise: 1) false positive membership inference results misled by similar examples; 2) membership inference methods are usually too complex for end users to understand and use. To address these issues, we propose an alternative insert-and-detect methodology, advocating that web users and content platforms employ unique identifiers for reliable and independent membership inference. Users and platforms can create their identifiers, embed them in copyrighted text, and independently detect them in future LLMs. As an initial demonstration, we introduce ghost sentences and a user-friendly last-$k$ words test, allowing end users to chat with LLMs for membership inference. Ghost sentences consist primarily of unique passphrases of random natural words, which can come with customized elements to bypass possible filter rules. The last-$k$ words test requires a significant repetition time of ghost sentences ($\ge10$). For cases with fewer repetitions, we designed an extra perplexity test, as LLMs exhibit high perplexity when encountering unnatural passphrases. We also conduct a comprehensive study on the memorization and membership inference of ghost sentences, examining factors such as training data scales, model sizes, repetition times, insertion positions, wordlist of passphrases, alignment, etc. Our study shows the possibility of applying ghost sentences in real scenarios and provides instructions for the potential application.

Introduction

As Large Language Models (LLMs) scale, they consume vast swaths of the internet, often including copyrighted books, code, and articles without explicit permission. Currently, creators face two major hurdles in proving their data was "scraped":

  1. False Positives: Existing Membership Inference Attacks (MIAs) often get confused by "similar" examples. A model might seem to know a specific poem simply because it read a thousand similar poems, not necessarily the user's specific work.
  2. Complexity: Most detection methods require deep expertise in statistics or coding, making them inaccessible to the average blogger or programmer.

This research proposes a proactive "insert-and-detect" framework to move from reactive detection to proactive watermarking, giving everyday users a transparent and reliable way to audit model training data.

Methodology: The Insert-and-Detect Framework

The core of this methodology is the Ghost Sentence—a unique identifier primarily composed of a "diceware" passphrase (a sequence of random natural words). Because these strings are mathematically improbable in natural language, their presence in a model's output serves as a "smoking gun" for training data inclusion.

The Ghost Sentence workflow: embedding a unique passphrase into text and using chat-based prompts to detect its presence in an LLM.

The paper details two primary detection strategies:

  • The Last-k Words Test: A user-friendly method where a user prompts an LLM with the beginning of their ghost sentence and asks it to complete the last $k$ words. If the model completes it verbatim, it has likely memorized the data.
  • The Perplexity Test: For cases where the data wasn't repeated enough for verbatim memorization, users can check the model's "perplexity" (a measure of how surprised the model is by a sequence). If the model finds a random ghost sentence "easy to predict," it is a strong indicator of prior exposure.

To ensure these identifiers are not filtered out by automated cleaning scripts, the authors suggest adding customizable prefixes to the passphrases to make them appear as natural (if slightly unusual) comments or footnotes.

Factors Influencing Memorization

The researchers evaluated their methodology across several diverse datasets, including Reddit posts (Webis-TLDR-17), patient-doctor conversations (HealthCareMagic), and code datasets. Key insights include:

  • The Memorization Threshold: Verbatim memorization usually requires $\ge 10$ repetitions. Memorization is a balance between the number of unique identifiers and the frequency of their repetition.
  • Position Matters: Ghost sentences are significantly more likely to be memorized if they appear in the latter half of a document. As auto-regressive models, LLMs use the preceding context to "anchor" the unique identifier.
  • Model Size and Alignment: Larger models (e.g., LLaMA-13B vs. 3B) are more effective at memorizing rare strings, making them easier to audit. Furthermore, safety alignment techniques like RLHF or DPO do not erase the memorization of ghost sentences; the watermark remains intact.

Key Results

The study demonstrates high statistical confidence in detection. Using a 10-word passphrase, a model correctly predicting even just the last 1 or 2 words results in a z-score so high that the probability of a "lucky guess" is nearly zero.

In experiments with OpenLLaMA-3B, 11 out of 16 users successfully identified their data. On average, these users contributed only about 24 examples each to a pool of 1.8 million documents—showing that ghost sentences are detectable even when they constitute only 0.0017% of the training tokens. For less-frequent data (7 repetitions), the perplexity test achieved a strong 0.891 ROC AUC.

Conclusion and Future Work

While model providers might eventually attempt to filter specific ghost sentence patterns, the authors argue that because these identifiers can be customized and embedded in diverse ways (like code comments or blog footers), making filtering "intractable" is possible. Future work aims to develop even more diverse unique identifiers that mimic natural language more closely to evade sophisticated filters while maintaining statistical uniqueness.

Summary updated

Abstract

A primary concern regarding training large language models (LLMs) is whether they abuse copyrighted online text. With the increasing training data scale and the prevalence of LLMs in daily lives, two problems arise: \textbf{1)} false positive membership inference results misled by similar examples; \textbf{2)} membership inference methods are usually too complex for end users to understand and use. To address these issues, we propose an alternative \textit{insert-and-detect} methodology, advocating that web users and content platforms employ \textbf{\textit{unique identifiers}} for reliable and independent membership inference. Users and platforms can create their identifiers, embed them in copyrighted text, and independently detect them in future LLMs. As an initial demonstration, we introduce \textit{\textbf{ghost sentences}} and a user-friendly last-$k$ words test, allowing end users to chat with LLMs for membership inference. Ghost sentences consist primarily of unique passphrases of random natural words, which can come with customized elements to bypass possible filter rules. The last-$k$ words test requires a significant repetition time of ghost sentences~($\ge10$). For cases with fewer repetitions, we designed an extra perplexity test, as LLMs exhibit high perplexity when encountering unnatural passphrases. We also conduct a comprehensive study on the memorization and membership inference of ghost sentences, examining factors such as training data scales, model sizes, repetition times, insertion positions, wordlist of passphrases, alignment, \textit{etc}. Our study shows the possibility of applying ghost sentences in real scenarios and provides instructions for the potential application.
BibTeX
@misc{zhao2024ghost,
  author = {Shuai Zhao and Linchao Zhu and Ruijie Quan and Yi Yang},
  title = {{Ghost Sentence: A Tool for Everyday Users to Copyright Data from Large Language Models}},
  year = {2024},
  eprint = {2403.15740},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2403.15740},
}
Figure