Abstract
As artificial intelligence advances rapidly, particularly with the advent of GANs and diffusion models, the accuracy of Image Inpainting Localization (IIL) has become increasingly challenging. Current IIL methods face two main challenges: a tendency towards overconfidence, leading to incorrect predictions; and difficulty in detecting subtle tampering boundaries in inpainted images. In response, we propose a new paradigm that treats IIL as a conditional mask generation task utilizing diffusion models. Our method, InpDiffusion, utilizes the denoising process enhanced by the integration of image semantic conditions to progressively refine predictions. During denoising, we employ edge conditions and introduce a novel edge supervision strategy to enhance the model's perception of edge details in inpainted objects. Balancing the diffusion model's stochastic sampling with edge supervision of tampered image regions mitigates the risk of incorrect predictions from overconfidence and prevents the loss of subtle boundaries that can result from overly stochastic processes. Furthermore, we propose an innovative Dual-stream Multi-scale Feature Extractor (DMFE) for extracting multi-scale features, enhancing feature representation by considering both semantic and edge conditions of the inpainted images. Extensive experiments across challenging datasets demonstrate that the InpDiffusion significantly outperforms existing state-of-the-art methods in IIL tasks, while also showcasing excellent generalization capabilities and robustness.
Introduction
The proliferation of sophisticated image inpainting techniques, driven by Generative Adversarial Networks (GANs) and diffusion models, has made the detection of manipulated image regions—Image Inpainting Localization (IIL)—a critical challenge in digital forensics. Traditional discriminative models often struggle with two major hurdles: overconfidence in erroneous predictions and the inability to delineate invisible boundaries where inpainted content seamlessly blends with the original background.
Methodology
InpDiffusion addresses these challenges by reframing IIL as a conditional mask generation task. Instead of a single-pass classification, the framework employs an iterative denoising process to progressively refine the localization mask from initial noise.
Conditional Diffusion Paradigm
The model utilizes an Adaptive Conditional Network (ACN) to guide the denoising process using the original image's semantic features. This stochastic approach acts as a buffer against the "all-or-nothing" overconfidence typical of standard CNNs, allowing the model to explore and settle into more accurate predictions.
Dual-stream Multi-scale Feature Extractor (DMFE)
To capture both the broad semantic context and fine-grained structural details, we introduce the DMFE. It utilizes parallel paths of dilated convolutions: one path progressively widens the field of view for global context, while the other focuses on local details to identify subtle inconsistencies.
Edge Supervision Strategy
A key innovation of InpDiffusion is the simultaneous prediction of tampered edges during the denoising process. By supervising the boundary of the tampered region, the model becomes significantly more sensitive to the minute "seams" left by inpainting tools, preventing the predicted mask from "bleeding" into the untampered background.
Results and Discussion
InpDiffusion was evaluated on several rigorous benchmarks, including Inpaint32K, DID, and real-world datasets like IMD and AutoSplice.
Performance and Generalization
The model achieved state-of-the-art (SOTA) results, with AUC scores reaching up to 96.2% on the Inpaint32K dataset. Notably, on the real-world IMD dataset, InpDiffusion outperformed the nearest competitor by a substantial 10.2% in AUC, demonstrating exceptional generalization beyond laboratory conditions.
Robustness
The framework maintains high accuracy even under common image "attacks" such as Gaussian blurring, noise injection, and resizing, which are often used to mask tampering traces.
Conclusion and Limitations
InpDiffusion demonstrates that generative models can significantly enhance the precision of forensic image analysis. While the iterative nature of diffusion models introduces higher computational costs compared to single-pass networks, the gains in accuracy and boundary sensitivity are substantial. Future work will focus on optimizing inference speed and scaling performance with larger pre-training datasets.
Summary updated