MarkTune: Improving the Quality-Detectability Trade-off in Open-Weight LLM Watermarking

Yizhou Zhao, Zhiwei Steven Wu, Adam BlockarXiv 2025 · DOI · arXiv

Abstract

MarkTune is a novel fine-tuning framework designed to embed robust, high-quality watermarks directly into the weights of open-source Large Language Models (LLMs). Unlike inference-time methods that can be easily bypassed by users, MarkTune utilizes on-policy reinforcement learning to optimize for watermark detectability while preserving model utility through regularization. Experimental results on Qwen3-4B and Llama2-7B demonstrate that MarkTune significantly outperforms existing weight-perturbation baselines like GaussMark, achieving a detection frontier comparable to state-of-the-art inference-time techniques without sacrificing text quality or reasoning capabilities.

Introduction

As open-weight models (such as Llama or Qwen) become the industry standard, ensuring their responsible use is a growing challenge. Traditional watermarking—which injects a hidden signal into text during the generation process—usually occurs at "inference time." However, because open-weight models allow users to control the entire generation pipeline, a malicious actor can simply disable the watermarking logic.

To address this, researchers have explored "model-embedded" watermarking, where the signal is baked into the model's weights. The current state-of-the-art, GaussMark, adds small amounts of Gaussian noise to weights to create a detectable bias in the output. However, this approach faces a harsh trade-off: increasing detectability typically requires perturbing weights to the point where text quality and reasoning capabilities suffer.

Methodology

MarkTune introduces a theoretically grounded framework that moves beyond simple weight noise. Instead of static perturbation, MarkTune uses on-policy fine-tuning to "tune" the watermark into the model weights while navigating the model's high-quality loss basin.

Comparison of inference-time watermarking vs. model-embedded watermarking (GaussMark and MarkTune).

The Fine-Tuning Framework

  1. The Reward: MarkTune treats the GaussMark detection statistic as a reward signal.
  2. The Optimization: It employs Group Relative Policy Optimization (GRPO), a critic-free reinforcement learning algorithm, to update model weights.
  3. The Constraint: To prevent quality degradation, the authors include a regularization term (either Cross-Entropy on human text or KL-divergence against an oracle model).

This approach allows MarkTune to find "watermark-sensitive directions" in the parameter space that increase detectability without compromising the model's fundamental capabilities.

Experimental Setup

  • Fine-tuning: Training was conducted on a subset of the OpenWebText dataset.
  • Evaluation: Generalization was tested on the C4 (RealNewsLike) split and the ELI5 dataset.
  • Downstream Benchmarks: Model utility was measured using MMLU (general knowledge), GSM8K (math), and MBPP (coding).

Key Results and Analysis

MarkTune demonstrates a superior "quality-detectability" trade-off compared to existing methods. On both Qwen3-4B and Llama2-7B, it pushed the detection frontier significantly further than GaussMark, matching the performance of inference-time methods like KGW while maintaining lower perplexity.

Quality-detectability trade-off for Qwen3-4B and Llama2-7B across different watermarking methods.

Robustness and Utility

MarkTune proved remarkably resilient to various removal attempts, maintaining high detectability after paraphrasing, round-trip translation, and even adversarial fine-tuning (up to 1,500 LoRA steps). Furthermore, utility preservation was high, with watermarked models showing negligible performance drops on standard benchmarks compared to unwatermarked baselines.

Discussion and Limitations

The success of MarkTune is attributed to the "flat basin" hypothesis of modern LLMs: the over-parameterized nature of these models allows for multiple weight configurations that yield high-quality output, some of which are more conducive to watermarking than others.

However, the method does involve a fine-tuning stage that requires GPU resources, unlike the near-instant perturbation of GaussMark. Future work will explore scaling this framework to larger models (70B+) and investigating more complex reward functions to further optimize the trade-off.

Summary updated

Abstract

Watermarking aims to embed hidden signals in generated text that can be reliably detected when given access to a secret key. Open-weight language models pose acute challenges for such watermarking schemes because the inference-time interventions that dominate contemporary approaches cannot be enforced once model weights are public. Existing watermaking techniques for open-weight models, such as the recently proposed GaussMark, typically rely on small modifications to model weights, which can yield signals detectable to those equipped with a secret key, but achieving detection power comparable to inference-time watermarks generally requires weight perturbations that noticeably reduce generation quality. We introduce MarkTune, a theoretically principled, on-policy fine-tuning framework that treats the GaussMark signal as a reward while simultaneously regularizing against degradation in text quality. We derive MarkTune as an improvement on GaussMark and demonstrate that MarkTune consistently improves the quality-detectability trade-off over GaussMark by steering finer-grained, watermark-aware weight updates within the model's representation space while preserving generation quality. Empirically, we show that MarkTune pushes the quality-detectability frontier of GaussMark close to that of inference-time watermarking, remains robust to paraphrasing and fine-tuning attacks, and exhibits strong generalization: a model fine-tuned on one dataset retains substantial watermark detection power on unseen datasets. Together, these results establish MarkTune as a general strategy for embedding robust, high-quality watermarks into open-weight LMs.
BibTeX
@misc{zhao2025marktune,
  author = {Yizhou Zhao and Zhiwei Steven Wu and Adam Block},
  title = {{MarkTune: Improving the Quality-Detectability Trade-off in Open-Weight LLM Watermarking}},
  year = {2025},
  eprint = {2512.04044},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2512.04044},
}
Figure