Evading Deepfake-Image Detectors with White- and Black-Box Attacks

Nicholas Carlini, Hany Farid

Abstract

It is now possible to synthesize highly realistic images of people who don't exist. Such content has, for example, been implicated in the creation of fraudulent social-media profiles responsible for disinformation campaigns. Significant efforts are, therefore, being deployed to detect synthetically-generated content. One popular forensic approach trains a neural network to distinguish real from synthetic content. We show that such forensic classifiers are vulnerable to a range of attacks that reduce the classifier to near-0% accuracy. We develop five attack case studies on a state-of-the-art classifier that achieves an area under the ROC curve (AUC) of 0.95 on almost all existing image generators, when only trained on one generator. With full access to the classifier, we can flip the lowest bit of each pixel in an image to reduce the classifier's AUC to 0.0005; perturb 1% of the image area to reduce the classifier's AUC to 0.08; or add a single noise pattern in the synthesizer's latent space to reduce the classifier's AUC to 0.17. We also develop a black-box attack that, with no access to the target classifier, reduces the AUC to 0.22. These attacks reveal significant vulnerabilities of certain image-forensic classifiers.

Introduction

As generative adversarial networks (GANs) like StyleGAN2 become capable of creating hyper-realistic faces of non-existent people, the "arms race" between fakes and forensic detectors has intensified. Recent forensic classifiers have shown impressive generalization, identifying fakes from architectures they weren't even trained on. However, these detectors are typically evaluated against "natural" variations like JPEG compression or blurring. This research addresses a critical security gap: adversarial robustness. We investigate whether a motivated attacker can intentionally modify a deepfake to bypass these detectors while keeping the image looking perfectly realistic to a human.

Attack Methodologies

The study evaluates two primary forensic classifiers (Wang et al. and Frank et al.) using five distinct attack case studies. These range from white-box attacks with full model access to black-box transfers where the attacker has no knowledge of the target model's parameters.

Adversarial perturbation process: (a) the resulting adversarial image, (b) the original synthetic image, and (c) the noise pattern scaled by 1/1000.

Key attack vectors include:

  • Distortion-minimizing (L2 and L0) Attacks: Using gradient descent to find the smallest possible change to an image that flips the classification from "fake" to "real."
  • Bit-Flipping (L0 refinement): A highly practical attack that only flips the least-significant bit of a pixel. This ensures the attack survives being saved as a PNG or even a high-quality JPEG.
  • Universal Adversarial Patch: Creating a single, small noise pattern (1% of the image area) that can be overlaid on any fake image to make it appear real to the detector.
  • Universal Latent-Space Attack: Instead of modifying pixels, the authors modify the GAN’s "latent space" (specifically the $w$ space in StyleGAN2). This "bakes" the adversarial nature into the image generation process itself.
  • Black-Box Attack: By training a "surrogate" detector, researchers showed that adversarial examples created for their model successfully transferred to fool the target model.

Key Insights and Sensitivity

Forensic detectors were found to be roughly 10 times more sensitive to adversarial perturbations than standard ImageNet classifiers. This suggests they are picking up on fragile, low-level pixel artifacts rather than robust semantic features.

Analysis of the perturbations revealed a grid-like pattern aligned with the $8 \times 8$ JPEG lattice, indicating that detectors might be over-relying on compression artifacts to identify fakes. Furthermore, the latent space vulnerability demonstrates that an attacker doesn't need to post-process an image; by finding a specific "adversarial attribute" in the GAN's latent space, they can generate an infinite number of different faces that appear "real" to the detector by default.

Results

The effectiveness of these attacks was devastating across all tested scenarios:

  • White-Box Success: On the Wang et al. detector, a bit-flipping attack on just 1% of pixels reduced the Area Under the ROC Curve (AUC) from 0.95 to 0.08.
  • Universal Patch: Adding a tiny noise patch to the corner of images reduced the AUC from 0.966 to 0.085.
  • Latent Space: The latent-space attack reduced StyleGAN2 detection AUC from 0.99 to 0.17.
  • Black-Box Transfer: Even without knowing the target model's weights, the transfer attack reduced the target's AUC from 0.96 to 0.22.
  • Real-World Application: In the "Andrew Walz" case study, a profile picture flagged as 87% fake was transformed into one classified as 99% real using a perceptually indistinguishable perturbation.

Conclusion

While these attacks focus on low-distortion perturbations, they highlight a fundamental weakness in current forensic methods. For deepfake detectors to be useful in high-stakes environments like social media verification, they must be trained using adversarial machine learning techniques, such as adversarial training or randomized smoothing. Simply being accurate on a static test set is insufficient to survive a motivated adversary in the evolving landscape of synthetic media.

Summary updated

Abstract

It is now possible to synthesize highly realistic images of people who don't exist. Such content has, for example, been implicated in the creation of fraudulent social-media profiles responsible for dis-information campaigns. Significant efforts are, therefore, being deployed to detect synthetically-generated content. One popular forensic approach trains a neural network to distinguish real from synthetic content. We show that such forensic classifiers are vulnerable to a range of attacks that reduce the classifier to near-0% accuracy. We develop five attack case studies on a state-of-the-art classifier that achieves an area under the ROC curve (AUC) of 0.95 on almost all existing image generators, when only trained on one generator. With full access to the classifier, we can flip the lowest bit of each pixel in an image to reduce the classifier's AUC to 0.0005; perturb 1% of the image area to reduce the classifier's AUC to 0.08; or add a single noise pattern in the synthesizer's latent space to reduce the classifier's AUC to 0.17. We also develop a black-box attack that, with no access to the target classifier, reduces the AUC to 0.22. These attacks reveal significant vulnerabilities of certain image-forensic classifiers.
BibTeX
@inproceedings{carlini2020evading,
  author = {Nicholas Carlini and Hany Farid},
  title = {{Evading Deepfake-Image Detectors with White- and Black-Box Attacks}},
  booktitle = {Workshops @ CVPR},
  year = {2020},
  url = {https://openreview.net/forum?id=c6PiXIJNV8},
}
Figure