Weakly-Supervised Deepfake Localization in Diffusion-Generated Images

Dragoș-Constantin Țânțaru, Elisabeta Oneață, Dan OneațăWACV 2024

TL;DR

This paper tackles the challenge of pinpointing manipulated regions within deepfake images generated by diffusion models, moving beyond simple "fake" or "real" labels. They introduce a weakly-supervised framework and a new dataset to systematically analyze how different factors like detection methods, supervision levels, and generators impact localization accuracy, finding that patch-based methods and generator choice are most critical.

What Problem Does This Paper Address?

The rise of powerful generative models, particularly diffusion models, has made it increasingly difficult to distinguish between real and fake images. While deepfake detection has focused on classifying entire images as real or fake, this binary output is often insufficient, especially for localized manipulations like face edits or inpainting. Current deepfake detection models are also largely trained and evaluated on images generated by older GAN architectures, leaving a gap in understanding their performance on newer diffusion-based fakes. Furthermore, most existing localization methods require explicit ground-truth masks, which are not always available, especially for novel manipulation techniques. This paper addresses the need for effective deepfake localization methods that can work with weaker supervision (image-level labels only) and are evaluated on modern diffusion-generated images.

Methods

The authors propose a weakly-supervised framework for deepfake localization. They adapt three existing categories of localization techniques, all using the Xception network as a common backbone:

  • GradCAM: Leverages explainability techniques to highlight regions most predictive of a "fake" label.
  • Patches (Patch-Forensics): Generates local patch scores that are then aggregated for classification. This method implicitly constructs localization maps.
  • Attention: Uses an attention mask to modulate feature maps, with the loss function encouraging the predicted mask to align with the image-level label.

They also consider a fully-supervised setting as an upper bound for performance. The core novelty lies in evaluating these methods under weakly-supervised conditions, where only image-level labels are available, and in their systematic analysis of various influencing factors.

Datasets

The paper introduces a new, detailed dataset comprising over 125,000 images. This dataset is designed to disentangle the effects of different factors:

  • Real Data: Subsets of CelebA-HQ and FFHQ datasets.
  • Fully-Synthesized Fake Data: Generated using the P2 diffusion model on CelebA-HQ and FFHQ.
  • Locally-Manipulated Fake Data: Created using the Repaint method (a diffusion-based inpainting technique) on CelebA-HQ and FFHQ. This includes manipulations like face swaps, attribute edits, and inpainting of specific facial regions (eyes, mouth, nose). They also generated local manipulations using other methods (Repaint-LDM, LaMa, Pluralistic) for cross-generator evaluation.

Key Insights and Ideas

  • Patch-based methods are robust: The "Patches" approach consistently outperformed GradCAM and Attention, especially in weakly-supervised settings. This suggests that focusing on local image patches is more effective for localization than global saliency maps or attention mechanisms when supervision is limited.
  • Weak supervision is feasible: The study demonstrates that deepfake localization is achievable even with only image-level labels, particularly for the Patches method. This is crucial for real-world scenarios where ground-truth masks are scarce.
  • Generator matters most: The type of generative model used to create the fake image had the most significant impact on localization performance, even more so than the level of supervision. This highlights the need for detectors to be robust to different generation techniques.
  • Latent diffusion models are challenging: Localizing manipulations generated by latent diffusion models (LDMs) proved to be particularly difficult, even in fully-supervised settings. This is attributed to the inpainting occurring in the latent space, which "hides" traces of manipulation during decoding.
  • Dataset mismatch hurts: Performance drops significantly when training and testing datasets come from different sources (e.g., CelebA-HQ vs. FFHQ), especially in weakly-supervised scenarios.

Key Results

  • In the weakly-supervised setting (Setup B: image-level label on locally manipulated images), the Patches method achieved an IoU of 37.7% and PBCA of 79.8% on the Repaint-P2/CelebA-HQ dataset.
  • In the fully-supervised setting (Setup C), Patches achieved a remarkable IoU of 83.7% and PBCA of 84.5%.
  • Detection performance (AP) was strong across methods in Setup B, suggesting that even with weak supervision, classifiers can be trained effectively on partially manipulated images.
  • Qualitative results showed Patches recovering manipulated regions better than GradCAM and Attention in weakly-supervised scenarios, though with some noisy activations.

Limitations and Future Work

  • Generalization to unseen datasets: While the paper explores generalization across generators, performance drops significantly when testing on datasets generated by different models, especially for latent diffusion models.
  • Dataset mismatch: The study highlights the sensitivity of models to dataset shifts, indicating a need for more robust generalization strategies.
  • Future work: The authors suggest extending this research to general-content images beyond faces and to other types of local manipulations generated by DDPMs. They also point to the challenge of localizing manipulations from latent diffusion models as a key area for future investigation.

Summary updated

BibTeX
@inproceedings{tantaru2024weakly,
  author = {Dragoș-Constantin Țânțaru and Elisabeta Oneață and Dan Oneață},
  title = {{Weakly-Supervised Deepfake Localization in Diffusion-Generated Images}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2024},
}
Figure