Introduction
As image editing tools become increasingly sophisticated, distinguishing between authentic and manipulated content has become a critical challenge. Current Image Manipulation Detection and Localization (IMDL) methods often struggle with scale variation, a lack of correlation modeling between pixels, and high false alarm rates on pristine images.
PSCC-Net addresses these hurdles with a high-speed, dual-path framework designed to detect and localize manipulations such as splicing, copy-move, and object removal. By integrating a coarse-to-fine progressive mask estimation with a novel Spatio-Channel Correlation Module (SCCM), it achieves state-of-the-art accuracy while maintaining real-time processing speeds of over 50 FPS for 1080p video.
Methodology
The PSCC-Net architecture consists of two main paths: a top-down feature extraction path and a bottom-up detection and localization path.
Top-Down Path (Feature Extraction)
The network utilizes HRNetV2p-W18 as its backbone. Unlike traditional backbones that downsample images sequentially, HRNet maintains high-resolution representations in parallel with low-resolution ones. This allows for dense cross-scale information exchange, ensuring that both local details and global context are preserved throughout the feature extraction process.
Bottom-Up Path (Detection & Localization)
The bottom-up path performs two simultaneous tasks:
- Detection: A dedicated Detection Head provides a binary score (pristine vs. manipulated). This explicit step significantly reduces false positives by allowing the model to "shut down" localization for images identified as authentic.
- Localization: Manipulation masks are generated in a progressive, coarse-to-fine fashion. A mask generated at a low resolution serves as a spatial prior for the next, higher-resolution stage, helping the network refine high-frequency details.
Spatio-Channel Correlation Module (SCCM)
The SCCM is the core innovation of PSCC-Net. At each scale in the bottom-up path, it captures both spatial and channel-wise correlations:
- Spatial Attention: Helps the network understand the global context of local features, determining if an object's appearance is consistent with its surroundings.
- Channel Attention: Enhances the representation of specific feature maps that are most indicative of tampering. To manage the memory requirements of spatial correlation, the authors employ a reshaping function to downscale the correlation matrix without losing essential feature information.
Experimental Results
PSCC-Net was trained on a synthetic dataset of approximately 0.38 million images. Despite using a smaller training set than some competitors, the model demonstrated superior performance across five standard benchmarks: Columbia, Coverage, CASIA, NIST16, and IMD20.
Localization and Detection Performance
PSCC-Net achieved state-of-the-art results in nearly all categories. On the Columbia dataset, it reached a pixel-level AUC of 98.2%, outperforming the previous best (SPAN) by 4.6%. In terms of detection accuracy on the CASIA-D benchmark, PSCC-Net achieved an AUC of 99.65%, a significant improvement over SPAN’s 67.33%.
Robustness and Efficiency
The model is highly resilient to common post-processing distortions, including JPEG compression (even at low quality factors like $q=50$), Gaussian blurring, and resizing. Furthermore, its lightweight backbone and progressive mechanism allow it to process 1080p images at 50+ FPS, making it suitable for real-time forensic applications.
Conclusion
PSCC-Net represents a significant advancement in IMDL by combining progressive refinement with explicit correlation modeling. While it excels on standard benchmarks, challenges remain with expertly crafted real-world forgeries where manipulation traces are meticulously hidden. Future work will focus on estimating mask uncertainty and improving the detection of complex, multi-scale copy-move attacks.
Summary updated