Frequency-aware GAN for Adversarial Manipulation Generation

Peifei Zhu, Genki Osada, Hirokatsu Kataoka, Tsubasa Takahashi

Abstract

As image editing tools become more sophisticated, the risk of misinformation grows. While deep-learning detectors can identify manipulated regions in images, they are often vulnerable to adversarial attacks. This paper introduces a Frequency-aware GAN designed for Adversarial Manipulation Generation (AMG). Unlike traditional attacks that target image-level classification, AMG aims to fool pixel-level semantic segmentation detectors. By incorporating frequency-domain features and high-frequency skip connections, the proposed method achieves a high attack success rate while maintaining superior visual quality and significantly faster processing speeds compared to traditional spatial attacks.

Introduction

Researchers have developed powerful deep-learning detectors to identify manipulated regions in images, but these models are vulnerable to adversarial attacks—small, intentional perturbations that trick a model into making incorrect predictions. Most existing attacks focus on simple classification (e.g., "is this image fake or real?"). This paper addresses the more challenging task of Adversarial Manipulation Generation (AMG), which requires fooling detectors that perform semantic segmentation to pinpoint exactly which pixels are manipulated.

The challenge is twofold: achieving pixel-level deception across both manipulated and authentic regions, and maintaining visual fidelity. Existing spatial attacks like PGD often introduce visible noise because they degrade high-frequency details like edges and textures, making the attack obvious to human observers.

Methodology

The authors propose a Frequency-aware GAN architecture designed to hide adversarial noise within the frequency components of an image.

The proposed Frequency-aware GAN architecture featuring DWT/IDWT layers and high-frequency skip connections.

Key components of the architecture include:

  • Frequency-Domain Integration: The model uses Discrete Wavelet Transform (DWT) to decompose features into low-frequency (LL) and high-frequency (LH, HL, HH) components.
  • Encoder-Decoder with Skip Connections: The generator utilizes an encoder-decoder structure with high-frequency skip connections that pass fine-grained details directly from the encoder to the decoder. This ensures the generated adversarial image preserves sharp edges and textures.
  • Optimal Loss Function: Training involves a combination of GAN loss (for visual realism) and a specialized Dice Loss to maximize the detector's error at the pixel level.

Experimental Setup

The method was evaluated on three benchmark datasets: DEFACTO (149k images), CASIAv2 (4,795 images), and COVER (100 images). The attacks targeted three state-of-the-art detectors: FCN, ManTra-Net, and MVSS-Net.

Results and Analysis

The proposed method consistently outperformed existing spatial and GAN-based attacks in both efficiency and effectiveness.

Quantitative comparison of attack methods across different datasets and detectors, highlighting the superior speed and attack success rate (ASR) of the proposed method.

Key Findings:

  • Superior Image Quality: The method achieved a PSNR of 40.1, significantly higher than FGSM (26.4) or PGD (30.4). Visually, the images remain nearly indistinguishable from the originals.
  • High Attack Success Rate (ASR): In white-box settings, the method achieved an ASR of over 60% across different datasets, consistently outperforming AdvGAN.
  • Efficiency: Unlike iterative attacks (like PGD) that must be recalculated for every image, this GAN-based approach generates perturbations in a single forward pass (approx. 0.01 seconds per image). As shown in the results table, the total run time for the DEFACTO dataset was reduced from 470 hours (PGD) to just 15 minutes.
  • The Frequency Gap: Visualization of high-frequency components confirmed that standard attacks cause massive degradation in these areas, which explains their visible noise. By preserving these components, the Frequency-aware GAN maintains high fidelity.

Conclusion

While the paper demonstrates a powerful attack in white-box scenarios, the success rate decreases in true black-box settings where the target model is unknown. However, the strong transferability observed suggests that AMG captures fundamental vulnerabilities in how models perceive forgery. This work serves as a call for the forensics community to develop more robust defenses, such as frequency-domain consistency checks, to protect detectors from imperceptible adversarial attacks.

Summary updated

Abstract

Image manipulation techniques have drawn growing concerns as manipulated images might cause morality and security problems. Various methods have been proposed to detect manipulations and achieved promising performance. However, these methods might be vulnerable to adversarial attacks. In this work, we design an Adversarial Manipulation Generation (AMG) task to explore the vulnerability of image manipulation detectors. We first propose an optimal loss function and extend existing attacks to generate adversarial examples. We observe that existing spatial attacks cause large degradation in image quality and find the loss of high-frequency detailed components might be its major reason. Inspired by this observation, we propose a novel adversarial attack that incorporates both spatial and frequency features into the GAN architecture to generate adversarial examples. We further design an encoder-decoder architecture with skip connections of high-frequency components to preserve fine details. We evaluated our method on three image manipulation detectors (FCN, ManTra-Net and MVSS-Net) with three benchmark datasets (DEFACTO, CASIAv2 and COVER). Experiments show that our method generates adversarial examples significantly fast (0.01s per image), preserves better image quality (PSNR 30% higher than spatial attacks), and achieves a high attack success rate. We also observe that the examples generated by AMG can fool both classification and segmentation models, which indicates better transferability among different tasks.
BibTeX
@inproceedings{zhu2023frequency,
  author = {Peifei Zhu and Genki Osada and Hirokatsu Kataoka and Tsubasa Takahashi},
  title = {{Frequency-aware GAN for Adversarial Manipulation Generation}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2023},
  doi = {10.1109/iccv51070.2023.00398},
}
Figure