ForgDiffuser: General Image Forgery Localization with Diffusion Models

Mengxi Wang, Shaozhang Niu, Jiwei ZhangIJCAI 2025 · DOI

Abstract

General Image Forgery Localization (GIFL) is critical for identifying tampered regions in images, yet traditional encoder-decoder models often suffer from decoder overconfidence and boundary ambiguity. We present ForgDiffuser, a novel framework that reformulates forgery localization as a conditional mask-generation task using diffusion models. By employing an iterative denoising process, ForgDiffuser effectively handles predictive uncertainty and captures fine-grained edge details. Our approach achieves state-of-the-art performance across multiple benchmarks, demonstrating significant improvements in mask completeness and boundary accuracy.

Introduction

The rapid advancement of image editing tools and AI-driven generation has made General Image Forgery Localization (GIFL) increasingly challenging. The goal is to precisely locate tampered areas, whether they result from traditional splicing and copy-move operations or modern AI synthesis.

Current state-of-the-art (SOTA) methods typically utilize standard encoder-decoder architectures. However, these models face two primary limitations:

  1. Decoder Overconfidence: Traditional decoders often produce binary masks that lack a mechanism for handling uncertainty, leading to incomplete or overly aggressive predictions.
  2. Boundary Ambiguity: As forgery techniques become more sophisticated, tampered regions blend seamlessly into backgrounds, making it difficult for models to capture the subtle "seams" at the edges.

Methodology

ForgDiffuser addresses these challenges by replacing the conventional one-shot decoder with a conditional diffusion process. Instead of a single-pass prediction, the model iteratively refines a mask from random noise, guided by features extracted from the suspect image.

The training framework of ForgDiffuser, integrating PVT features with AGM and BDM modules into a conditional diffusion process.

The architecture consists of several key components:

  • Attention-Guided Module (AGM): Utilizing a Joint Attention (JA) mechanism, the AGM aggregates multi-scale features from a PVTv2 backbone. This allows the model to balance low-level local artifacts with high-level semantic inconsistencies.
  • Boundary-Driven Module (BDM): The BDM explicitly supervises the learning of edge distributions, generating an edge map that guides the final mask to align with physical boundaries.
  • Global-Local Consistency Fusion (GLCF): To stabilize the stochastic nature of diffusion during inference, GLCF fuses multiple predicted masks based on their global and local variance, filtering out uncertain noise.

Experimental Setup

We evaluated ForgDiffuser on six diverse benchmark datasets:

  • Traditional Image Forgery (TIF): CASIA1 and RLS26K (splicing/copy-move), and DID (inpainting).
  • AI-Generated Image Forgery (AIGIF): Auto and BSN (DALL-E2 and BrushNet generations).
  • Real-World Challenges: The IMD dataset, which contains complex, real-world manipulations.

Results and Insights

ForgDiffuser achieved SOTA results on five of the six datasets. Notably, on the challenging IMD dataset, it improved the F1-score by 0.05 and IoU by 0.04 over the previous best model, EC-Net.

Qualitative comparison between ForgDiffuser and state-of-the-art methods across various forgery types.

Key insights from our evaluation include:

  • Diffusion as a Regularizer: The step-by-step denoising process prevents the model from committing to large, incorrect predictions, effectively mitigating decoder overconfidence.
  • Feature Synergy: Combining texture clues from the BDM with contextual clues from the AGM is essential for accurate localization.
  • Robustness: ForgDiffuser maintains high accuracy even under common image attacks like Gaussian or Salt & Pepper noise, where baseline models often fail.

Conclusion

ForgDiffuser represents a significant step forward in image forensics by leveraging the probabilistic strengths of diffusion models. While the iterative nature of diffusion introduces higher inference latency compared to one-shot decoders, the gains in accuracy and boundary precision are substantial. Future research will focus on optimizing inference speed and extending this framework to video forgery localization.

Summary updated

BibTeX
@inproceedings{wang2025forgdiffuser,
  author = {Mengxi Wang and Shaozhang Niu and Jiwei Zhang},
  title = {{ForgDiffuser: General Image Forgery Localization with Diffusion Models}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.24963/ijcai.2025/218},
}
Figure