The Vulnerability and Fragility of Neural Network Watermarks

567 concepts320 papers1 proposal

Deep neural networks are expensive to train, making them highly valuable intellectual property. To protect these digital assets from unauthorized distribution and theft, researchers developed model watermarking—embedding secret triggers, out-of-distribution behaviors, or precise weight perturbations to verify ownership.

However, a growing body of research reveals a fundamental flaw: these watermarks are incredibly fragile. They frequently disappear during standard model optimization or targeted removal attacks. To understand why watermarks are so easy to wash away, we must look at how they interact with a network's underlying optimization landscape.

The Outlier Problem: Why Fine-Tuning Erases Watermarks

To understand why watermarks fail, we must look at how they are embedded. Many schemes, such as the backdoor-based approach proposed by Yossi Adi et al., force a model to memorize arbitrary, out-of-distribution inputs (e.g., classifying a random abstract pattern as a specific, unrelated label). Because these triggers are completely unrelated to the primary task, the network treats them as statistical outliers.

When an adversary fine-tunes a stolen model on clean, legitimate data, the network undergoes parameter drift. In a phenomenon akin to catastrophic forgetting, the model prioritizes minimizing loss on the new, meaningful data distribution, naturally discarding the low-utility, overfitted watermark circuits. As Chenchen Gu et al. demonstrate, weights-based watermarks are highly susceptible to erasure when a model undergoes additional training on non-watermarked datasets.

Furthermore, this erasure does not require massive resources. Xuankai Liu et al. showed that an attacker with access to as little as 10% to 20% of the original training data can effectively scrub backdoor watermarks using targeted fine-tuning and data augmentation. In many cases, aggressive fine-tuning with high learning rates can eliminate the hidden signature entirely before the model's performance on its primary task even begins to degrade, as highlighted by Mohammed Lansari et al..

The Filtering Effect of Model Extraction

What happens when an attacker doesn't steal the weights directly, but instead copies the model's behavior via API queries? This is known as a model extraction or distillation attack.

Traditional trigger-set watermarks fail completely in this scenario due to a natural filtering effect. Because the watermark triggers are highly overfitted anomalies, a user querying the model with typical, in-distribution inputs will never activate them. As a result, the stolen "surrogate" model only learns the core functional logic of the original network.

Masoumeh Shafieinejad et al. found that black-box model stealing effectively strips watermarks because the extracted model fails to inherit these ungeneralized triggers. Similarly, Jialuo Chen et al. confirmed that invasive watermarks do not transfer to cloned models during black-box extraction, rendering the protection useless against API-based theft.

Weight-Space Disruption: Pruning, Quantization, and Merging

Other watermarking strategies attempt to avoid the pitfalls of behavioral triggers by embedding signatures directly into the network's parameter values or numerical precision. However, these weight-intrinsic watermarks are highly vulnerable to structural optimizations.

When a model is deployed to edge devices, it typically undergoes compression techniques like pruning (removing low-magnitude weights) or quantization (reducing numerical precision). Sam Leroux et al. note that these standard lifecycle operations frequently corrupt or completely erase embedded watermarks because the watermark relies on fine-grained parameter configurations that compression algorithms discard as noise.

An even greater challenge arises from weight-space operations like model merging, where multiple models are blended together to combine their capabilities. In a comprehensive study, Tianshuo Cong demonstrated that quantization- and weight-based watermarks are easily "washed away" during merging. Because merging algorithms average, trim, or drop parameter values to resolve interference between models, the delicate numerical signatures required for ownership verification are destroyed. As Futa Kai Waseda et al. explain, when a fingerprinted model's weights are diluted by being averaged with several other models, the specific input-output associations drop well below the detection threshold.

Reverse Engineering and No-Box Evasion

Even when developers attempt to hide the watermarking mechanism through security by obscurity, attackers can often bypass or reverse-engineer the defense by exploiting the statistical footprint of the watermark itself.

For instance, Thibaud Gloaguen et al. showed that statistical probing of watermarked language models allows an observer to accurately estimate internal parameters like context window size and secret key length for minimal computational cost. Once these parameters are exposed, designing a targeted removal or spoofing attack becomes trivial.

Even in "no-box" settings where the attacker has absolutely no access to the watermark decoder or its outputs, generative models remain vulnerable. Xiangman Li et al. demonstrated that watermarks can be stripped from diffusion models using simple post-processing blurring or by fine-tuning the generator with a surrogate decoder, proving that hiding the verification architecture does not guarantee security against structural model modifications.

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 324 papers
  2. Version 1 · 2026-10-04 · 215 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_watermark_fragility_7ed266a2,
  title = {watermark fragility},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/watermark-fragility/rdr_7ed266a2a7214eb5bdfbded412195236},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 2, frozen text):

@misc{lacuna_watermark_fragility_7ed266a2_v2,
  title = {watermark fragility},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/watermark-fragility/rdr_7ed266a2a7214eb5bdfbded412195236/v/2},
  note = {Version 2, frozen 2026-10-04. Accessed 2026-11-19},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake