Evading Deepfake-Image Detectors with White- and Black-Box Attacks

Nicholas Carlini, Hany Farid

Abstract

It is now possible to synthesize highly realistic images of people who do not exist. Such content has, for example, been implicated in the creation of fraudulent social-media profiles responsible for disinformation campaigns. Significant efforts are, therefore, being deployed to detect synthetically-generated content. One popular forensic approach trains a neural network to distinguish real from synthetic content. We show that such forensic classifiers are vulnerable to a range of attacks that reduce the classifier to near-0% accuracy. We develop five attack case studies on a state-of-the-art classifier that achieves an area under the ROC curve (AUC) of 0.95 on almost all existing image generators, when only trained on one generator. With full access to the classifier, we can flip the lowest bit of each pixel in an image to reduce the classifier's AUC to 0.0005; perturb 1% of the image area to reduce the classifier's AUC to 0.08; or add a single noise pattern in the synthesizer's latent space to reduce the classifier's AUC to 0.17. We also develop a black-box attack that, with no access to the target classifier, reduces the AUC to 0.22. These attacks reveal significant vulnerabilities of certain image-forensic classifiers.

Introduction

As generative models like StyleGAN2 become capable of creating hyper-realistic faces of non-existent people, the research community has responded by building "forensic classifiers." These neural networks are trained to distinguish between real photos and synthetically generated ones. However, neural networks are notoriously brittle. While these detectors perform well on standard benchmarks, this research investigates whether they are robust against an adversary actively trying to fool them. This paper addresses the critical lack of adversarial robustness in current deepfake detection pipelines, demonstrating that state-of-the-art detectors are highly vulnerable to imperceptible perturbations.

Methods and Attack Strategies

The study evaluates the robustness of prominent forensic classifiers (such as those by Wang et al. and Frank et al.) using several distinct attack strategies:

  1. Distortion-minimizing Attack (White-box): Employs gradient descent and Lagrangian relaxation to find the minimal $\ell_2$ or $\ell_0$ norm change needed to flip a prediction.
  2. $\ell_0$ Bit-flipping: A refined attack where only the least-significant bit of a small percentage of pixels is flipped. This ensures the image remains visually identical and can be saved as a standard PNG/JPEG without losing the adversarial effect.
  3. Universal Adversarial Patch: A single 24x24 pixel noise patch is generated. When overlaid on any fake image, it causes the detector to label it as "real."
  4. Universal Latent-Space Attack: Instead of modifying a finished image, the input latent vector ($w$) of the StyleGAN2 generator is modified, forcing the generator to produce images that appear "real" to the detector from the moment of creation.
  5. Black-Box Transfer Attack: Assuming no access to the target detector's weights, a "surrogate" detector is trained to generate adversarial examples that are then transferred to the original target.
Adversarial perturbation process: an original synthetic image (b) is combined with a scaled noise pattern (c) to create an adversarial image (a) that is misclassified as real by forensic detectors.

Datasets

The study utilizes several key datasets for training and evaluation:

  • Wang et al. Dataset: 94,036 images (half real, half ProGAN-generated) used to evaluate the primary target detector.
  • ProGAN & StyleGAN2: Images generated from these architectures were used to test the attacks.
  • Custom Surrogate Dataset: For the black-box attack, 1 million ProGAN images and 1 million real images were used to train a ResNet-18 surrogate model.

Analysis of Vulnerabilities

The research reveals that forensic detectors are roughly 10 times more sensitive to adversarial perturbations than standard ImageNet classifiers. A tiny change that wouldn't affect an object recognizer can completely break a deepfake detector. This sensitivity suggests that detectors are likely overfitting to low-level synthesis artifacts, such as the 8x8 JPEG-like lattice introduced during the GAN upsampling process, rather than learning semantic facial features.

Furthermore, the success of the latent-space attack suggests that the distribution of images a GAN can produce is wide enough to include many images that a detector considers "real," even if they are entirely synthetic. The black-box results prove that "security through obscurity" fails; an attacker can build a similar model and transfer attacks with high success.

Experimental Results

  • White-Box Destruction: By flipping the lowest bit of just 1% of pixels, the detector's Area Under the ROC Curve (AUC) dropped from 0.95 to 0.0005.
  • Universal Patch Success: A single 1% area noise patch reduced the AUC to 0.085.
  • Latent Space Impact: Modifying the StyleGAN2 latent space reduced the detector's AUC from 0.99 to 0.17.
  • Black-Box Transferability: Even without knowing the target model's parameters, the transfer attack reduced the target's AUC from 0.96 to 0.22.

Conclusion

While adversarial training or randomized smoothing could offer some defense, these measures are often bypassed by newer, stronger attacks. Heavy "image laundering" (resizing, aggressive re-compression) can sometimes strip away adversarial noise, but this also tends to reduce the detector's original accuracy. Forensic researchers must move beyond simple binary classifiers and design models inherently capable of withstanding adversarial manipulation.

Summary updated

Abstract

It is now possible to synthesize highly realistic images of people who do not exist. Such content has, for example, been implicated in the creation of fraudulent social-media profiles responsible for dis-information campaigns. Significant efforts are, therefore, being deployed to detect synthetically-generated content. One popular forensic approach trains a neural network to distinguish real from synthetic content. We show that such forensic classifiers are vulnerable to a range of attacks that reduce the classifier to near-0% accuracy. We develop five attack case studies on a state-of-the-art classifier that achieves an area under the ROC curve (AUC) of 0.95 on almost all existing image generators, when only trained on one generator. With full access to the classifier, we can flip the lowest bit of each pixel in an image to reduce the classifier's AUC to 0.0005; perturb 1% of the image area to reduce the classifier's AUC to 0.08; or add a single noise pattern in the synthesizer's latent space to reduce the classifier's AUC to 0.17. We also develop a black-box attack that, with no access to the target classifier, reduces the AUC to 0.22. These attacks reveal significant vulnerabilities of certain image-forensic classifiers.
BibTeX
@inproceedings{carlini2020evading,
  author = {Nicholas Carlini and Hany Farid},
  title = {{Evading Deepfake-Image Detectors with White- and Black-Box Attacks}},
  booktitle = {Workshops @ CVPR},
  year = {2020},
  url = {https://openreview.net/forum?id=c6PiXIJNV8},
}
Figure