Diff-IFL: Towards General Image Forgery Localization using Diffusion Probabilistic Model

Ping Xu, Jiangqun Ni, Jian Zhang, Yulin Zhang, Shiyuan Tang

Abstract

With wide applications of image editing tools, forged images have become a great public concern. Although existing methods for image forgery localization (IFL) could achieve fairly good results on several public datasets, most of them perform unsatisfactorily for cross-dataset evaluation and online social network applications. To tackle this issue, a novel coarse-to-fine framework using Diffusion probabilistic model for Image Forgery Localization (Diff-IFL) is proposed in this paper, which consists of a coarse localization module and a mask diffusion module. The coarse localization module employs a transformer-based architecture to represent the tampered images and generate coarse masks. While the mask diffusion module formulates IFL as a mask reconstruction task, it relies on the extracted forgery feature representations as the conditional prior to gradually recover the clean ground-truth mask from the noisy mask. Extensive experiments demonstrate that Diff-IFL outperforms other SOTA methods and exhibits superior robustness against social media forgery.
BibTeX
@inproceedings{xu2024diff,
  author = {Ping Xu and Jiangqun Ni and Jian Zhang and Yulin Zhang and Shiyuan Tang},
  title = {{Diff-IFL: Towards General Image Forgery Localization using Diffusion Probabilistic Model}},
  booktitle = {ICME},
  year = {2024},
  url = {https://openreview.net/forum?id=3zKLbVoSOR},
}

Loading related content...

Figure