Yet Another Watermark for Large Language Models

Siyuan Bao, Ying Shi, Zhiguang Yang, Hanzhou Wu, Xinpeng ZhangarXiv 2025 · arXiv · OpenReview

Abstract

Existing watermarking methods for large language models (LLMs) mainly embed watermark by adjusting the token sampling prediction or post-processing, lacking intrinsic coupling with LLMs, which may significantly reduce the semantic quality of the generated marked texts. Traditional watermarking methods based on training or fine-tuning may be extendable to LLMs. However, most of them are limited to the white-box scenario, or very time-consuming due to the massive parameters of LLMs. In this paper, we present a new watermarking framework for LLMs, where the watermark is embedded into the LLM by manipulating the internal parameters of the LLM, and can be extracted from the generated text without accessing the LLM. Comparing with related methods, the proposed method entangles the watermark with the intrinsic parameters of the LLM, which better balances the robustness and imperceptibility of the watermark. Moreover, the proposed method enables us to extract the watermark under the black-box scenario, which is computationally efficient for use. Experimental results have also verified the feasibility, superiority and practicality. This work provides a new perspective different from mainstream works, which may shed light on future research.

Introduction

As Large Language Models (LLMs) become ubiquitous, distinguishing AI-generated content from human writing has become a critical security and ethical challenge. Existing watermarking techniques generally fall into two categories: post-processing (replacing words or structures after generation) and logit manipulation (biasing token sampling). While effective, these methods often degrade text fluency and semantic quality because they operate externally to the model's internal logic. This paper introduces a novel approach that embeds watermarks directly into the model's internal parameters, creating a signal that is "entangled" with the model’s intrinsic dynamics.

Methodology: Parameter-Level Embedding

The core innovation is a structured, sparse manipulation of the LLM's final output linear layer. Unlike standard logit bias approaches that add a fixed value during sampling, this method scales the actual weights ($W$) and biases ($b$) of the output layer.

Using a secret key and a pseudo-random function, the vocabulary is partitioned into a "selected" subset (the watermark signal). Weights for these "green-list" tokens are amplified by a factor $\alpha_{\uparrow} \ge 1$, while others are scaled down. Because this scaling is applied to the weights that interact with the hidden state $h$, the watermark is dynamically coupled with the linguistic context.

Impact of scaling factor $\alpha$ on the trade-off between perplexity (PPL) and detectability (z-score).

Detection remains a "black-box" process. A verifier only needs the secret key to identify the "green-list" tokens and performs a statistical z-score test on the generated text to confirm the presence of the watermark.

Key Results

The researchers evaluated the method using LLaMA3-8B on the C4 dataset. The results demonstrate a significant advantage in maintaining text quality while ensuring high detectability.

Performance comparison of the proposed method against baseline and state-of-the-art watermarking techniques.

As shown in the table above, the proposed method achieved a Perplexity (PPL) of 3.93, which is remarkably close to the un-watermarked baseline of 3.82. In contrast, previous state-of-the-art methods like Kirchenbauer et al. resulted in a much higher PPL of 5.08, indicating a noticeable drop in naturalness. Despite its subtlety, the method maintains a strong z-score of 6.49, well above the standard detection threshold required for high confidence.

Robustness and Insights

The "Aha!" moment of this research is the realization that logit-based watermarking is a rigid subset of parameter modulation. By moving the intervention into the weights, the watermark becomes part of the model's generation dynamics. This leads to "distribution smoothness"—preserving the natural flow and style of the LLM without the "spiky" probability distributions common in hard logit biasing.

Furthermore, the watermark proved robust against common text attacks. Even when subjected to 50% random word masking, deletion, or insertion, the z-score remained high enough for reliable identification, demonstrating the method's practicality in real-world scenarios where text might be edited.

Conclusion

While the embedding process requires white-box access to model weights, the resulting watermark is highly imperceptible and easily detectable in a black-box setting. Future work may explore more complex modulation strategies and extension to multi-modal models. This parameter-level approach provides a promising new perspective for securing LLM outputs without sacrificing the quality that makes them useful.

Summary updated

Abstract

Existing watermarking methods for large language models (LLMs) mainly embed watermark by adjusting the token sampling prediction or post-processing, lacking intrinsic coupling with LLMs, which may significantly reduce the semantic quality of the generated marked texts. Traditional watermarking methods based on training or fine-tuning may be extendable to LLMs. However, most of them are limited to the white-box scenario, or very time-consuming due to the massive parameters of LLMs. In this paper, we present a new watermarking framework for LLMs, where the watermark is embedded into the LLM by manipulating the internal parameters of the LLM, and can be extracted from the generated text without accessing the LLM. Comparing with related methods, the proposed method entangles the watermark with the intrinsic parameters of the LLM, which better balances the robustness and imperceptibility of the watermark. Moreover, the proposed method enables us to extract the watermark under the black-box scenario, which is computationally efficient for use. Experimental results have also verified the feasibility, superiority and practicality. This work provides a new perspective different from mainstream works, which may shed light on future research.
BibTeX
@misc{bao2025yet,
  author = {Siyuan Bao and Ying Shi and Zhiguang Yang and Hanzhou Wu and Xinpeng Zhang},
  title = {{Yet Another Watermark for Large Language Models}},
  year = {2025},
  eprint = {2509.12574},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2509.12574},
}
Figure