Abstract
Traditional image forgery localization (IFL) often simplifies the detection problem into a binary classification task, failing to account for the multi-source nature of modern composite images. We present SAFIRE (Segment Any Forged Image Region), a novel framework that shifts the paradigm toward "source partitioning." By leveraging point prompting and region-based contrastive learning, SAFIRE identifies image regions based on their unique origins. Our approach achieves state-of-the-art performance on standard binary benchmarks and introduces the first capability for multi-source forgery partitioning, offering enhanced interpretability for forensic analysis.
Introduction
Image Forgery Localization (IFL) has historically been treated as a binary segmentation task, where pixels are labeled as either authentic or forged. While this binary approach works for simple splices, it is increasingly inadequate for modern "Franken-images"—complex composites that may combine elements from multiple donor photos, AI-generated content, and inpainted regions.
Furthermore, binary models often rely on detecting specific artifacts, such as JPEG compression traces or camera sensor noise. This makes them brittle when faced with heavy post-processing or novel forgery techniques. SAFIRE addresses these limitations by partitioning an image into distinct source regions based on their underlying consistency, rather than just searching for forgery traces.
Methodology
Inspired by the Segment Anything Model (SAM), SAFIRE utilizes point prompting to define and segment source regions. The framework consists of four primary components:
- Region-to-Region Contrastive Pretraining: The image encoder is pretrained to ensure that features from the same source region are clustered together in a high-dimensional space, while features from different sources are pushed apart. This establishes a baseline of "source consistency."
- Point-Prompted Segmentation: During training, the model is provided with a single point and tasked with segmenting the entire region sharing that point's origin. To facilitate this with binary datasets, the authors developed a "Point Mask" strategy that treats connected components as distinct regions.
- Area-Adaptive Loss: To mitigate the common issue of models ignoring small forged objects, SAFIRE employs a weighted loss function that assigns higher importance to smaller segments.
- Automated Inference via Clustering: For fully automated forensic analysis, SAFIRE generates a grid of points (e.g., 16x16) across the image. The resulting features are clustered using algorithms like k-means or DBSCAN. The most confident mask from each cluster is then selected and aggregated to form the final multi-source or binary prediction map.
Datasets and Evaluation
To evaluate SAFIRE, the authors utilized five established benchmarks: Columbia, COVERAGE, CocoGlide, RealisticTampering, and NC16. Additionally, they introduced two new datasets:
- SafireMS-Auto: A large-scale training dataset (~123,000 images) created by automatically splicing regions from pristine photos using SAM-generated masks.
- SafireMS-Expert: A manually curated evaluation dataset of 238 images containing 2 to 4 distinct source regions, specifically designed to test multi-source partitioning capabilities.
Key Results
SAFIRE demonstrates superior performance across both traditional and novel forensic tasks. On standard binary IFL benchmarks, it outperformed top-tier models such as TruFor and MVSS-Net, achieving an average F1 score of 62.6% at a fixed threshold and 73.0% at the best threshold.
Beyond binary detection, SAFIRE is the first method capable of explaining how an image was composed by separating different donor images into distinct, color-coded regions. On the SafireMS-Expert dataset, it achieved a permuted mIoU of 90.3% for 2-source images. Furthermore, the model proved highly robust, maintaining significantly higher accuracy than competitors when subjected to Gaussian blur, noise, and JPEG compression.
Conclusion and Future Work
SAFIRE redefines the scope of image forensics by moving beyond binary labels to source-aware partitioning. While the point-wise mask decoding adds some computational complexity compared to single-pass networks, the gains in interpretability and robustness are substantial. Future research may extend this point-prompting approach to video forensics or integrate it with large-scale image databases to facilitate provenance filtering and original source identification.
Summary updated