Feature Extraction Matters More: Universal Deepfake Disruption through Attacking Ensemble Feature Extractors

Long Tang, Dengpan Ye, Zhenhao Lu, Yunming Zhang, Shengshan Hu, Yue Xu, Chuanxi ChenarXiv 2023 · DOI · arXiv

Abstract

As deepfake technology advances, protecting facial privacy requires proactive measures. This paper presents FOUND (Feature-Output ensemble UNiversal Disruptor), an active defense mechanism that generates imperceptible adversarial perturbations to "poison" images before they are processed by deepfake models. Unlike previous methods that target the final output, FOUND focuses on disrupting the feature extraction stage common to most GAN-based architectures. This approach results in a universal disruptor that is nearly 10 times faster to generate and significantly more effective at rendering deepfake outputs unusable across various models.

Introduction: The Need for Active Defense

Deepfake models, primarily based on Generative Adversarial Networks (GANs), pose significant risks to individual privacy by enabling the creation of realistic but fraudulent facial manipulations. While passive detection methods attempt to identify fakes after the fact, active defense aims to prevent their creation entirely. By adding a carefully crafted "digital poison pill"—an adversarial perturbation—to a clean image, users can ensure that any attempt to manipulate their face results in a garbled, non-human output.

The primary challenge in active defense is achieving universality. Different deepfake models (e.g., StarGAN, AttGAN, HiSD) use distinct architectures and target different facial attributes. A perturbation designed for one model often fails to transfer to others. FOUND addresses this by identifying a common bottleneck: the feature extraction process.

The FOUND Framework

The researchers propose a two-stage disruption process designed to maximize transferability and efficiency.

Stage 1: Feature-Ensemble Disruption

The core insight of FOUND is that while deepfake models vary in their generation techniques, they rely on similar feature extraction modules to understand the input face. Stage 1 targets these "roots" by aggregating feature maps from multiple models. By maximizing the Mean Square Error (MSE) between original and adversarial features and using Kullback-Leibler (KL) divergence to spread the disruption across model distributions, FOUND prevents the attack from overfitting to a single architecture.

Stage 2: Output Disruption with Gradient-Ensemble

To further degrade the final image, a second stage attacks the end-to-end generation process. To handle the conflicting optimization goals of different models, FOUND introduces a Gradient-Ensemble strategy. Instead of simple loss averaging, it treats each model's gradient as a separate task and fuses them to find a consensus direction for the perturbation.

The FOUND framework consists of a two-stage process: Stage 1 targets feature extraction modules, while Stage 2 employs a Gradient-Ensemble strategy for output disruption.

Experimental Results

The FOUND method was evaluated against several state-of-the-art deepfake models, including StarGAN, AGGAN, AttGAN, and HiSD, using datasets like CelebA and LFW.

Efficiency and Effectiveness

FOUND demonstrates a massive leap in efficiency. It can generate a universal disruptor in just 0.71 hours on a single GPU, whereas the previous state-of-the-art, CMUA-Watermark, required over 7 hours and three GPUs. Despite this speed, FOUND achieved a higher success rate ($SR_{mask}$) of 98.55% on the CelebA dataset.

Visual Disruption Quality

Quantitatively, FOUND significantly increases the Fréchet Inception Distance (FID), indicating a high degree of "unreality" in the resulting deepfakes. For instance, against StarGAN, FOUND pushed the FID to 423.96, compared to CMUA’s 329.09. Qualitatively, the outputs are visibly more distorted, often appearing as unrecognizable patterns rather than faces.

Comparison of deepfake outputs across different models. FOUND consistently produces more severe disruption and garbled results compared to the CMUA baseline.

Transferability and Liveness Detection

FOUND also exhibits strong black-box transferability, effectively disrupting models it was not trained on, such as StarGAN-v2. Furthermore, images protected by FOUND are highly likely to fail automated "liveness" checks, providing an additional layer of security against identity spoofing.

Conclusion

By shifting the focus from the "branches" (specific attributes) to the "roots" (feature extraction), FOUND provides a robust and efficient solution for universal deepfake disruption. Its ability to generate high-quality protection in a fraction of the time of previous methods makes it a practical tool for large-scale facial privacy protection.

Summary updated

Abstract

Adversarial example is a rising way of protecting facial privacy security from deepfake modification. To prevent massive facial images from being illegally modified by various deepfake models, it is essential to design a universal deepfake disruptor. However, existing works treat deepfake disruption as an End-to-End process, ignoring the functional difference between feature extraction and image reconstruction, which makes it difficult to generate a cross-model universal disruptor. In this work, we propose a novel Feature-Output ensemble UNiversal Disruptor (FOUND) against deepfake networks, which explores a new opinion that considers attacking feature extractors as the more critical and general task in deepfake disruption. We conduct an effective two-stage disruption process. We first disrupt multi-model feature extractors through multi-feature aggregation and individual-feature maintenance, and then develop a gradient-ensemble algorithm to enhance the disruption effect by simplifying the complex optimization problem of disrupting multiple End-to-End models. Extensive experiments demonstrate that FOUND can significantly boost the disruption effect against ensemble deepfake benchmark models. Besides, our method can fast obtain a cross-attribute, cross-image, and cross-model universal deepfake disruptor with only a few training images, surpassing state-of-the-art universal disruptors in both success rate and efficiency.
BibTeX
@misc{tang2023feature,
  author = {Long Tang and Dengpan Ye and Zhenhao Lu and Yunming Zhang and Shengshan Hu and Yue Xu and Chuanxi Chen},
  title = {{Feature Extraction Matters More: Universal Deepfake Disruption through Attacking Ensemble Feature Extractors}},
  year = {2023},
  eprint = {2303.00200},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2303.00200},
}
Figure