GumbelSoft: Diversified Language Model Watermarking via the GumbelMax-trick

Jiayi Fu, Xuandong Zhao, Ruihan Yang, Yuansen Zhang, Jiangjie Chen, Yanghua Xiao

Introduction

As Large Language Models (LLMs) become integrated into daily workflows, the ability to distinguish machine-generated text from human writing is paramount. Decoding-based watermarking has emerged as a robust solution, embedding statistical signals into text during generation. However, state-of-the-art methods like the GumbelMax-trick (specifically the Exponential Watermark) suffer from a significant drawback: determinism. When prompted with the same input, these models produce identical, "robotic" responses, failing to capture the natural variability of human language.

The repetition problem in current watermarked LLMs and proposed solution paths.

Addressing the Determinism Problem

The root of this lack of diversity lies in the fixed nature of the decoder function and the pseudo-random function used to generate watermark keys. In standard GumbelMax watermarking, the decoder always selects the token that maximizes a Gumbel-adjusted score. This lack of variety frustrates users and makes AI interactions feel less natural.

To overcome this, we introduce GumbelSoft, a framework that shifts from deterministic selection to a sampling-based approach. GumbelSoft is built upon a simplified Logits-Addition framework. Instead of complex exponential transformations, GumbelSoft directly adds Gumbel-distributed noise to the model's output logits. This approach is mathematically equivalent to previous methods but offers a more direct path to introducing uncertainty.

The GumbelSoft Mechanism

The core innovation of GumbelSoft is replacing the standard argmax operation with sampling from a softmax distribution. By introducing a temperature parameter ($\tau$), the model can select tokens that are statistically likely but not necessarily the absolute maximum. This ensures that even when a non-optimal token is chosen, it still carries a significant watermark signal, maintaining high detectability while breaking the cycle of repetition.

Experimental Evaluation

The effectiveness of GumbelSoft was evaluated using Llama2-7b and Llama2-7b-chat models across diverse tasks:

  • Text Completion: Evaluated on the C4 dataset.
  • Question Answering: Evaluated using the Alpaca dataset.
  • Diversity Analysis: Measured using 20 high-entropy prompts repeated 50 times to calculate Self-BLEU scores.

Key Results and Insights

GumbelSoft demonstrates a superior balance between text diversity and watermark strength. While traditional GumbelMax watermarks yield a Self-BLEU score of 1.0 (indicating zero diversity), GumbelSoft significantly reduces this score, allowing for varied and natural-sounding outputs.

AUROC vs. Self-BLEU trade-off, showing GumbelSoft's superior performance in balancing detectability and diversity.

Detectability and Robustness

In high-diversity scenarios, GumbelSoft’s detection accuracy (measured by AUROC) consistently outperformed existing methods by margins of 0.1 to 0.3. Furthermore, the method proved resilient to "T5-span attacks," where an external model attempts to rewrite and "scrub" the watermark. The strength of the Gumbel signal allows it to survive significant textual edits better than traditional methods like KGW. Importantly, this is achieved without sacrificing text quality; perplexity scores remained nearly identical to those of unwatermarked models.

Conclusion

GumbelSoft effectively resolves the determinism flaw in high-performance LLM watermarking. By integrating softmax sampling into the GumbelMax framework, it provides a "Pareto optimal" solution that preserves the statistical strength of the watermark while restoring the natural diversity of generated text. Future work will explore the mathematical properties of the softmax variant more deeply and test its performance on complex reasoning tasks.

Summary updated

Abstract

Large language models (LLMs) excellently generate human-like text, but also raise concerns about misuse in fake news and academic dishonesty. Decoding-based watermark, particularly the watermark based on the GumbelMax trick (GM watermark), is a standout solution for safeguarding machine-generated texts due to its notable detectability. However, GM watermark encounters a major challenge with generation diversity, always yielding identical outputs for the same prompt, negatively impacting generation diversity and user experience. To overcome this limitation, we introduce a new type of GM watermark, the Logits-Addition watermark, as well as three variants that aim to enhance diversity, particularly the GumbelSoft watermark (i.e., the softmax variant of the Logits-Addition watermark). When assessed for detectability in high diversity settings, our Gumbelsoft demonstrates superior performance, with its AUROC score exceeding those of the two alternative variants by a margin of 0.1 to 0.3 and outperforming other decoding-based watermarking methods by a minimum of 0.1.
BibTeX
@inproceedings{fu2024gumbelsoft,
  author = {Jiayi Fu and Xuandong Zhao and Ruihan Yang and Yuansen Zhang and Jiangjie Chen and Yanghua Xiao},
  title = {{GumbelSoft: Diversified Language Model Watermarking via the GumbelMax-trick}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2024},
  doi = {10.18653/v1/2024.acl-long.315},
}
Figure