Abstract
MarkTune is a novel fine-tuning framework designed to embed robust, high-quality watermarks directly into the weights of open-source Large Language Models (LLMs). Unlike inference-time methods that can be easily bypassed by users, MarkTune utilizes on-policy reinforcement learning to optimize for watermark detectability while preserving model utility through regularization. Experimental results on Qwen3-4B and Llama2-7B demonstrate that MarkTune significantly outperforms existing weight-perturbation baselines like GaussMark, achieving a detection frontier comparable to state-of-the-art inference-time techniques without sacrificing text quality or reasoning capabilities.
Introduction
As open-weight models (such as Llama or Qwen) become the industry standard, ensuring their responsible use is a growing challenge. Traditional watermarking—which injects a hidden signal into text during the generation process—usually occurs at "inference time." However, because open-weight models allow users to control the entire generation pipeline, a malicious actor can simply disable the watermarking logic.
To address this, researchers have explored "model-embedded" watermarking, where the signal is baked into the model's weights. The current state-of-the-art, GaussMark, adds small amounts of Gaussian noise to weights to create a detectable bias in the output. However, this approach faces a harsh trade-off: increasing detectability typically requires perturbing weights to the point where text quality and reasoning capabilities suffer.
Methodology
MarkTune introduces a theoretically grounded framework that moves beyond simple weight noise. Instead of static perturbation, MarkTune uses on-policy fine-tuning to "tune" the watermark into the model weights while navigating the model's high-quality loss basin.
The Fine-Tuning Framework
- The Reward: MarkTune treats the GaussMark detection statistic as a reward signal.
- The Optimization: It employs Group Relative Policy Optimization (GRPO), a critic-free reinforcement learning algorithm, to update model weights.
- The Constraint: To prevent quality degradation, the authors include a regularization term (either Cross-Entropy on human text or KL-divergence against an oracle model).
This approach allows MarkTune to find "watermark-sensitive directions" in the parameter space that increase detectability without compromising the model's fundamental capabilities.
Experimental Setup
- Fine-tuning: Training was conducted on a subset of the OpenWebText dataset.
- Evaluation: Generalization was tested on the C4 (RealNewsLike) split and the ELI5 dataset.
- Downstream Benchmarks: Model utility was measured using MMLU (general knowledge), GSM8K (math), and MBPP (coding).
Key Results and Analysis
MarkTune demonstrates a superior "quality-detectability" trade-off compared to existing methods. On both Qwen3-4B and Llama2-7B, it pushed the detection frontier significantly further than GaussMark, matching the performance of inference-time methods like KGW while maintaining lower perplexity.
Robustness and Utility
MarkTune proved remarkably resilient to various removal attempts, maintaining high detectability after paraphrasing, round-trip translation, and even adversarial fine-tuning (up to 1,500 LoRA steps). Furthermore, utility preservation was high, with watermarked models showing negligible performance drops on standard benchmarks compared to unwatermarked baselines.
Discussion and Limitations
The success of MarkTune is attributed to the "flat basin" hypothesis of modern LLMs: the over-parameterized nature of these models allows for multiple weight configurations that yield high-quality output, some of which are more conducive to watermarking than others.
However, the method does involve a fine-tuning stage that requires GPU resources, unlike the near-instant perturbation of GaussMark. Future work will explore scaling this framework to larger models (70B+) and investigating more complex reward functions to further optimize the trade-off.
Summary updated