PSCC-Net: Progressive Spatio-Channel Correlation Network for Image Manipulation Detection and Localization

Xiaohong Liu, Yaojie Liu, Jun Chen, Xiaoming Liu

Introduction

As image editing tools become increasingly sophisticated, distinguishing between authentic and manipulated content has become a critical challenge. Current Image Manipulation Detection and Localization (IMDL) methods often struggle with scale variation, a lack of correlation modeling between pixels, and high false alarm rates on pristine images.

PSCC-Net addresses these hurdles with a high-speed, dual-path framework designed to detect and localize manipulations such as splicing, copy-move, and object removal. By integrating a coarse-to-fine progressive mask estimation with a novel Spatio-Channel Correlation Module (SCCM), it achieves state-of-the-art accuracy while maintaining real-time processing speeds of over 50 FPS for 1080p video.

Methodology

The PSCC-Net architecture consists of two main paths: a top-down feature extraction path and a bottom-up detection and localization path.

The PSCC-Net architecture featuring a top-down path for feature extraction and a bottom-up path for progressive mask generation and manipulation detection.

Top-Down Path (Feature Extraction)

The network utilizes HRNetV2p-W18 as its backbone. Unlike traditional backbones that downsample images sequentially, HRNet maintains high-resolution representations in parallel with low-resolution ones. This allows for dense cross-scale information exchange, ensuring that both local details and global context are preserved throughout the feature extraction process.

Bottom-Up Path (Detection & Localization)

The bottom-up path performs two simultaneous tasks:

  1. Detection: A dedicated Detection Head provides a binary score (pristine vs. manipulated). This explicit step significantly reduces false positives by allowing the model to "shut down" localization for images identified as authentic.
  2. Localization: Manipulation masks are generated in a progressive, coarse-to-fine fashion. A mask generated at a low resolution serves as a spatial prior for the next, higher-resolution stage, helping the network refine high-frequency details.

Spatio-Channel Correlation Module (SCCM)

The SCCM is the core innovation of PSCC-Net. At each scale in the bottom-up path, it captures both spatial and channel-wise correlations:

  • Spatial Attention: Helps the network understand the global context of local features, determining if an object's appearance is consistent with its surroundings.
  • Channel Attention: Enhances the representation of specific feature maps that are most indicative of tampering. To manage the memory requirements of spatial correlation, the authors employ a reshaping function to downscale the correlation matrix without losing essential feature information.

Experimental Results

PSCC-Net was trained on a synthetic dataset of approximately 0.38 million images. Despite using a smaller training set than some competitors, the model demonstrated superior performance across five standard benchmarks: Columbia, Coverage, CASIA, NIST16, and IMD20.

Qualitative comparison of PSCC-Net against SPAN and Ground Truth (GT) across various manipulation types.

Localization and Detection Performance

PSCC-Net achieved state-of-the-art results in nearly all categories. On the Columbia dataset, it reached a pixel-level AUC of 98.2%, outperforming the previous best (SPAN) by 4.6%. In terms of detection accuracy on the CASIA-D benchmark, PSCC-Net achieved an AUC of 99.65%, a significant improvement over SPAN’s 67.33%.

Robustness and Efficiency

The model is highly resilient to common post-processing distortions, including JPEG compression (even at low quality factors like $q=50$), Gaussian blurring, and resizing. Furthermore, its lightweight backbone and progressive mechanism allow it to process 1080p images at 50+ FPS, making it suitable for real-time forensic applications.

Conclusion

PSCC-Net represents a significant advancement in IMDL by combining progressive refinement with explicit correlation modeling. While it excels on standard benchmarks, challenges remain with expertly crafted real-world forgeries where manipulation traces are meticulously hidden. Future work will focus on estimating mask uncertainty and improving the detection of complex, multi-scale copy-move attacks.

Summary updated

BibTeX
@article{liu2022pscc,
  author = {Xiaohong Liu and Yaojie Liu and Jun Chen and Xiaoming Liu},
  title = {{PSCC-Net: Progressive Spatio-Channel Correlation Network for Image Manipulation Detection and Localization}},
  journal = {IEEE Trans. Circuits Syst. Video Technol.},
  year = {2022},
  url = {https://openreview.net/forum?id=CLrAvVIOZ5t},
}
Figure