New Evaluation Metrics Capture Quality Degradation due to LLM Watermarking

Karanpartap Singh, James Zou

TL;DR

This paper reveals that current LLM watermarking techniques are far less subtle than previously thought. By introducing two new evaluation metrics—LLM-as-a-judge and embedding-based binary classification—the researchers demonstrate that watermarking significantly degrades text coherence and leaves a detectable "fingerprint" that simple classifiers can identify without needing the watermark's secret key.

What Problem Does This Paper Address?

As Large Language Models (LLMs) become ubiquitous, "watermarking"—the practice of subtly biasing a model's word choices to make its output identifiable—has emerged as a key tool for tracing AI-generated content and preventing plagiarism.

However, the field has a measurement problem. Most researchers evaluate watermark quality using "perplexity" (how predictable the text is to a model) or "n-gram diversity." The authors argue these metrics are too blunt; they can’t tell if a paragraph is logically sound or if the writing style has become repetitive and shallow. Furthermore, watermarks are often assumed to be "invisible" to third parties who don't have the secret key. This paper challenges that assumption, asking: Does watermarking actually hurt the user experience, and can an outsider tell the difference?

Methods

The authors propose two "black-box" evaluation mechanisms that don't require knowledge of the underlying watermarking algorithm:

  1. Automated Judgement with GPT: Inspired by the "LLM-as-a-judge" trend, they used GPT-3.5-Turbo and GPT-4 to perform head-to-head comparisons between watermarked and unwatermarked versions of the same prompt. The "judge" scored the text on a 1–5 Likert scale across seven criteria: relevance, depth of detail, clarity, coherence/logic, originality, use of examples, and accuracy.
  2. Binary Classifier on Embeddings: The researchers took the text outputs and converted them into high-dimensional vectors using OpenAI’s text-embedding-ada-002. They then trained a simple 4-layer Multi-Layer Perceptron (MLP) to distinguish between watermarked and unwatermarked text. If the classifier succeeds, it proves the watermark has fundamentally altered the statistical distribution of the text in a way that is detectable even without the "secret key."

Datasets

The study evaluated three distinct types of content to ensure generalizability:

  • LongForm (Wikipedia): 251 documents covering various general-interest topics.
  • C4-RealNewsLike: 500 samples of text extracted from online news articles.
  • Scientific Papers (arXiv/PubMed): 252 abstracts from structured academic documents.

All tests were performed using the Llama-2-7B model as the base generator.

Key Insights and Ideas

  • The "Subtlety" Myth: The most striking finding is that watermarks are not invisible. A simple neural network could distinguish watermarked text with roughly 71% accuracy (and up to 80% for certain methods). This suggests that watermarking leaves a "semantic signature" that third parties could potentially use to bypass or identify watermarked content.
  • Coherence is the First Casualty: When the LLM judge preferred unwatermarked text, the biggest reason cited was coherence and logical flow. Watermarking forces the model to choose "green-listed" tokens that might be statistically probable but contextually awkward, leading to a "degradation in the depth of the response."
  • "Distortion-Free" Isn't Free: The researchers tested a "Robust Distortion-Free" watermarking method designed to maintain the original model's probability distribution. Surprisingly, the LLM judge still preferred the unwatermarked version 67.5% of the time, proving that even mathematically "neutral" watermarks can perceptibly harm writing quality.
  • The Robustness-Quality Trade-off: There is a direct correlation between how easy a watermark is to detect and how much it ruins the text. As the "strength" ($\delta$) of the watermark increases, detection accuracy nears 90%, but the gap in quality scores between watermarked and clean text widens significantly.

Key Results

  • Human-like Preference: Across all datasets, the GPT-judger preferred unwatermarked text 67.4% of the time, while watermarked text was preferred only 25.5% of the time.
  • Classifier Performance: The MLP classifier achieved an average AUC of 0.75 and 71% accuracy across all datasets. On the "Distortion-Free" watermark, accuracy climbed even higher, exceeding 80%.
  • Specific Degradation: In head-to-head scores, unwatermarked text consistently beat watermarked text by a wide margin in "Coherence," "Depth," and "Use of Examples," while "Accuracy" remained relatively stable.

Limitations and Future Work

The authors acknowledge a few constraints:

  • Model Dependency: The study relies heavily on GPT-3.5 and GPT-4 as the "gold standard" for quality. While these models are highly capable, they may have their own biases.
  • Sample Size: Some experiments used relatively small subsets (N=200-500), which might not capture the full diversity of all possible LLM prompts.
  • Detection Evasion: The paper doesn't explore how a malicious actor might "scrub" a watermark (e.g., by paraphrasing) before the classifier sees it.

Future Directions: The authors suggest that future watermarking research must move beyond perplexity and focus on maintaining semantic integrity. They also propose using more sophisticated classifiers to further investigate the "detectability" of these supposedly invisible markers.

Summary updated

Abstract

With the increasing use of large-language models (LLMs) like ChatGPT, watermarking has emerged as a promising approach for tracing machine-generated content. However, research on LLM watermarking often relies on simple perplexity or diversity-based measures to assess the quality of watermarked text, which can mask important limitations in watermarking. Here we introduce two new easy-to-use methods for evaluating watermarking algorithms for LLMs: 1) evaluation by LLM-judger with specific guidelines; and 2) binary classification on text embeddings to distinguish between watermarked and unwatermarked text. We apply these methods to characterize the effectiveness of current watermarking techniques. Our experiments, conducted across various datasets, reveal that current watermarking methods are detectable by even simple classifiers, challenging the notion of watermarking subtlety. We also found, through the LLM judger, that watermarking impacts text quality, especially in degrading the coherence and depth of the response. Our findings underscore the trade-off between watermark robustness and text quality and highlight the importance of having more informative metrics to assess watermarking quality.
BibTeX
@misc{singh2023new,
  author = {Karanpartap Singh and James Zou},
  title = {{New Evaluation Metrics Capture Quality Degradation due to LLM Watermarking}},
  year = {2023},
  eprint = {2312.02382},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2312.02382},
}
Figure