Are Watermarks Bugs for Deepfake Detectors? Rethinking Proactive Forensics

Xiaoshuai Wu, Xin Liao, Bo Ou, Yuling Liu, Zheng QinIJCAI 2024

Abstract

Proactive forensics, such as invisible watermarking, is essential for tracking the provenance of AI-generated images. However, these watermarks often act as "bugs" for passive deepfake detectors by smoothing out the very forgery artifacts these detectors rely on. This paper introduces AdvMark, a framework that transforms watermarks into "helpful" adversarial perturbations. By fine-tuning the watermark encoder against a surrogate detector, AdvMark ensures that watermarked deepfakes are correctly identified while maintaining high traceability and visual quality.

1. Introduction

The rise of deepfakes has led to two primary defensive strategies: passive forensics, which detects statistical anomalies in images, and proactive forensics, which embeds tracking information at the source. A significant, yet overlooked, conflict exists between these methods. Standard robust watermarks are typically optimized for natural images; when applied to deepfakes, they can inadvertently mask the subtle signals of manipulation, causing detectors to misclassify fakes as real.

Figure 1: Comparison of deepfake detection performance under different watermarking scenarios. Standard watermarking (b) often degrades detection accuracy, while AdvMark (c) enhances it by treating the watermark as a helpful adversarial signal.

As shown in Figure 1, current watermarking techniques can significantly drop the accuracy of deepfake detectors. The authors identify this as a critical safety loophole and propose a "helpful adversarial" approach to bridge the gap between provenance tracking and forgery detection.

2. The AdvMark Framework

The AdvMark framework addresses this conflict through a two-stage training procedure that repurposes the watermark signal to assist, rather than hinder, passive detectors.

Stage I: Robust Watermark Training

In the first stage, a standard encoder-decoder pair is trained to embed and extract watermark bits ($w$) from an image ($x$). A noise layer is utilized during training to ensure the watermark remains resilient to common image distortions like JPEG compression or Gaussian noise. This ensures the primary goal of proactive forensics—traceability—is met.

Stage II: Adversarial Fine-tuning

The core innovation of AdvMark is the adversarial fine-tuning stage. The authors freeze a "surrogate" deepfake detector and fine-tune the watermark encoder using a specialized Fooling Loss ($\mathcal{L}_D$). Unlike traditional adversarial attacks that aim to deceive a model into making a mistake, this "helpful" attack optimizes the watermark to nudge the image's representation across the decision boundary toward the correct ground-truth label (e.g., pushing a watermarked fake image deeper into the "fake" classification region).

Figure 2: The AdvMark training pipeline. Stage I establishes a robust watermark, while Stage II fine-tunes the encoder to produce helpful adversarial signals that assist deepfake detectors.

3. Experimental Evaluation

The researchers evaluated AdvMark using a comprehensive suite of facial data and forgery techniques:

  • Datasets: Real faces were sourced from CelebA-HQ. Forgeries included face swapping (SimSwap), expression reenactment (FOMM), attribute editing (StarGAN), and entire synthesis (StyleGAN).
  • Detectors: The framework was tested against nine well-known deepfake detectors, including Xception, EfficientNet, and CNND, to assess both white-box and black-box performance.

4. Results and Analysis

The quantitative improvements provided by AdvMark are striking. While baseline watermarking methods (like MBRS or SepMark) often dropped detection accuracy to below 20%, AdvMark restored and even enhanced performance.

  • Detection Accuracy: In white-box settings, AdvMark achieved nearly 100% detection accuracy. In black-box scenarios (unseen detectors), the "helpfulness" of the signal transferred effectively, especially when using an ensemble of surrogate detectors during training, which raised average accuracy to over 95%.
  • Traceability and Quality: Crucially, these gains were achieved without breaking the watermark. The Bit Error Rate (BER) for provenance extraction remained near zero. Furthermore, the visual quality of the images remained high, with PSNR and SSIM values comparable to standard non-adversarial watermarking.

5. Conclusion

AdvMark reconciles the tension between proactive and passive forensics by recognizing that invisible watermarks are structurally similar to adversarial perturbations. By harnessing this vulnerability for defensive purposes, the authors demonstrate that watermarks can be transformed from "bugs" into "features" that strengthen the overall AI safety ecosystem. Future work will explore extending this concept to more granular forensic tasks, such as tamper localization.

Summary updated

BibTeX
@inproceedings{wu2024are,
  author = {Xiaoshuai Wu and Xin Liao and Bo Ou and Yuling Liu and Zheng Qin},
  title = {{Are Watermarks Bugs for Deepfake Detectors? Rethinking Proactive Forensics}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2024},
}
Figure