Evaluating the security of public surrogate watermark detectors

Chloé Imadache, Eva Giboulot, Teddy FuronICASSP 2025 · DOI

TL;DR

This paper proposes a novel approach to make digital watermarking more practical by creating "surrogate" detectors that can be publicly shared. This addresses the security risk of exposing the true detector, allowing attackers to easily bypass watermarks. The research investigates how to train these surrogates and quantifies the trade-off between their accuracy and the security they provide against sophisticated attacks.

What Problem Does This Paper Address?

Digital watermarking is a crucial technology for content traceability, especially with the rise of generative AI and the potential for deepfakes and misinformation. However, a major hurdle to widespread adoption is the need to keep the watermark detector secret. If the detector is public, attackers can use "white-box" or "black-box" attacks to craft adversarial examples that fool the detector, rendering the watermark useless. This paper tackles this challenge by proposing a solution: a publicly accessible "surrogate" detector that mimics the behavior of a more accurate, but private, ground-truth detector. The core problem is to understand how much information can leak from this surrogate detector and how effectively attackers can leverage it to compromise the private detector.

Methods

The paper introduces a method for training these surrogate watermark detectors, which are implemented as deep neural networks. The key idea is "distillation," where the surrogate network learns to mimic the output probabilities (p-values) of the private, ground-truth detector. Two strategies for distillation are explored: "full distillation," where the surrogate learns from the private detector's output on perturbed images, and "original p-value," where it learns from the private detector's output on the original, unperturbed images. The researchers also investigate "key augmentation," where the surrogate is trained on images watermarked with various keys to improve its robustness.

To assess the security of these surrogates, the paper proposes a method to transfer attacks from the public surrogate detector to the private detector. This involves using powerful white-box attacks on the surrogate and then scaling the resulting adversarial perturbation to fool the private detector.

Datasets

The primary dataset used for training and evaluation is a subset of the MIRFlickR 1M dataset, consisting of 10,000 images. For the Stable-Signature watermarking scheme, a specific set of "generated" images was created by encoding and then decoding images from the MIRFlickR dataset using Stable-Diffusion's VAE. This was done to ensure the detector learned the watermarking signal rather than just distinguishing between real and generated images.

Key Insights and Ideas

  • The "Surrogate" Solution: The core idea of using a less accurate but publicly shareable surrogate detector is a pragmatic approach to balance usability and security in watermarking.
  • Distillation Trade-offs: The choice of distillation strategy (full vs. original p-value) significantly impacts the surrogate's behavior and its vulnerability to attacks. Using the original p-value seems to offer better security by leaking less information.
  • Augmentation is Key: Employing image augmentations and multiple keys during surrogate training is crucial for improving its robustness against various attacks.
  • Attack Transferability is Real: Adversarial attacks crafted against the public surrogate detector can indeed be transferred to compromise the private, ground-truth detector, though the effectiveness varies.
  • Watermarking Scheme Matters: Different watermarking techniques (TrustMark vs. Stable Signature) exhibit varying levels of robustness against attacks, even when using the same surrogate training strategies.

Key Results

The paper demonstrates that surrogate detectors can be trained effectively with high classification accuracy (close to 100% for distinguishing watermarked from cover images). The "Augmentations | Multiple keys" training strategy generally provided the best defense against attacks on both TrustMark and Stable Signature watermarking schemes.

However, the research also highlights that even with these strategies, surrogate detectors are not perfectly sound, meaning their p-value distributions deviate from the ideal uniform distribution under the null hypothesis. This deviation suggests that while they can be trained to be robust, they might still leak information that attackers can exploit.

Limitations and Future Work

A key limitation acknowledged is that adapting the strength of attacks to evaluate transferability requires access to the private detector, which is excluded in the attacker's scenario. The authors suggest that their loss function might favor reducing errors on watermarked images, potentially leading to less ideal p-value distributions for cover images.

Future work could focus on developing better methods for transferring attacks and distillation techniques that leak less information to the attacker. Investigating how to improve the soundness of surrogate detectors and exploring more sophisticated attack transfer mechanisms are also promising directions.

Summary updated

BibTeX
@inproceedings{imadache2025evaluating,
  author = {Chloé Imadache and Eva Giboulot and Teddy Furon},
  title = {{Evaluating the security of public surrogate watermark detectors}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2025},
  doi = {10.1109/icassp49660.2025.10889821},
}
Figure