PSCC-Net: Progressive Spatio-Channel Correlation Network for Image Manipulation Detection and Localization

Xiaohong Liu, Yaojie Liu, Jun Chen, Xiaoming Liu

TL;DR

PSCC-Net is a high-speed, dual-path framework designed to detect and localize image manipulations like splicing and object removal. By combining a coarse-to-fine progressive mask estimation with a novel Spatio-Channel Correlation Module (SCCM), it achieves state-of-the-art accuracy while processing 1080p video at over 50 FPS.

What Problem Does This Paper Address?

As image editing tools become more sophisticated, distinguishing between "authentic" and "manipulated" content is increasingly difficult. Current Image Manipulation Detection and Localization (IMDL) methods face three primary hurdles:

  1. Scale Variation: Tampered areas can range from a few pixels to nearly the entire image. Standard encoder-decoder models often struggle to balance local details with global context.
  2. Lack of Correlation Modeling: Many models try to map an image directly to a binary mask without considering how pixels relate to one another. This often leads to overfitting on specific attack types.
  3. High False Alarm Rates: Most existing research assumes an input image is manipulated. When presented with a pristine image, these models often "hallucinate" forgeries, making them unreliable for real-world systems.

Methods

PSCC-Net tackles these issues through a two-path architecture:

  • Top-Down Path (Feature Extraction): The network uses HRNetV2p-W18 as a backbone. Unlike traditional backbones that downsample images sequentially, HRNet maintains high-resolution representations in parallel with low-resolution ones, allowing for dense cross-scale information exchange.
  • Bottom-Up Path (Detection & Localization): This path performs two tasks. First, a Detection Head provides a binary score (pristine vs. manipulated) to reduce false positives. Second, it generates manipulation masks in a progressive, coarse-to-fine fashion. A mask generated at a low resolution serves as a "prior" or guide for the next, higher-resolution stage.
  • Spatio-Channel Correlation Module (SCCM): This is the core innovation. At each scale in the bottom-up path, the SCCM captures both spatial and channel-wise correlations.
  • Spatial Attention helps the network understand the global context of local features (e.g., "does this car look like it belongs on this road?").
  • Channel Attention enhances the representation of specific feature maps that are most indicative of tampering.
  • To manage the massive memory requirements of spatial correlation, the authors use a reshaping function ($h$) to downscale the correlation matrix without losing feature information.

Datasets

The authors trained PSCC-Net on a synthetic dataset of approximately 0.38M images, covering splicing, copy-move, and removal (inpainting). This is notably smaller than the datasets used by competitors like ManTra-Net (1.25M images).

For evaluation, they used five standard benchmarks:

  • Columbia & Coverage: Focused on splicing and copy-move.
  • CASIA & NIST16: Large-scale datasets with various manipulation types.
  • IMD20: A challenging real-world dataset of manipulated images collected from the internet.

Key Insights and Ideas

  • Progressive Refinement is Key: By starting with a coarse mask and refining it, the network avoids the "difficulty" of predicting high-frequency details from scratch. The lower-resolution masks act as an attention mechanism for the higher-resolution stages.
  • Explicit Detection Reduces Noise: By including a dedicated detection head at the start of the bottom-up path, the model can effectively "shut down" localization for pristine images, drastically reducing false alarms compared to models that only output masks (like SPAN).
  • Efficiency Matters: Using a light-weight backbone and a progressive mechanism allows the model to run at 50+ FPS on 1080p images. This makes it viable for real-time applications, whereas previous SOTA methods like ManTra-Net are significantly slower.

Key Results

  • Localization Performance: PSCC-Net achieved SOTA results across almost all benchmarks. For example, on the Columbia dataset, it reached a pixel-level AUC of 98.2%, outperforming the previous best (SPAN) by 4.6%.
  • Detection Accuracy: On the CASIA-D detection benchmark, PSCC-Net achieved an AUC of 99.65%, a massive jump from SPAN’s 67.33%.
  • Robustness: The model proved highly resilient to common post-processing distortions like JPEG compression, Gaussian blurring, and resizing. It was particularly effective against JPEG compression, maintaining high performance even at low quality factors ($q=50$).

Limitations and Future Work

While PSCC-Net performs well on benchmarks, the authors noted limitations when dealing with expertly crafted real-world forgeries (found in the IMD20 dataset). In cases where manipulation traces are elaborately removed or where the same pattern is copied at multiple different scales, the model may only localize parts of the forged region.

Future work will likely focus on estimating the uncertainty of predicted masks to provide more nuanced feedback to human investigators and further improving the model's ability to handle complex, multi-scale copy-move attacks.

Summary updated

Abstract

To defend against manipulation of image content, such as splicing, copy-move, and removal, we develop a Progressive Spatio-Channel Correlation Network (PSCC-Net) to detect and localize image manipulations. PSCC-Net processes the image in a two-path procedure: a top-down path that extracts local and global features and a bottom-up path that detects whether the input image is manipulated, and estimates its manipulation masks at multiple scales, where each mask is conditioned on the previous one. Different from the conventional encoder-decoder and no-pooling structures, PSCC-Net leverages features at different scales with dense cross-connections to produce manipulation masks in a coarse-to-fine fashion. Moreover, a Spatio-Channel Correlation Module (SCCM) captures both spatial and channel-wise correlations in the bottom-up path, which endows features with holistic cues, enabling the network to cope with a wide range of manipulation attacks. Thanks to the light-weight backbone and progressive mechanism, PSCC-Net can process <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$1,080\text{P}$ </tex-math></inline-formula> images at 50+FPS. Extensive experiments demonstrate the superiority of PSCC-Net over the state-of-the-art methods on both detection and localization. Codes and models are available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/proteus1991/PSCC-Net</uri> .
BibTeX
@article{liu2022pscc,
  author = {Xiaohong Liu and Yaojie Liu and Jun Chen and Xiaoming Liu},
  title = {{PSCC-Net: Progressive Spatio-Channel Correlation Network for Image Manipulation Detection and Localization}},
  journal = {IEEE Trans. Circuits Syst. Video Technol.},
  year = {2022},
  url = {https://openreview.net/forum?id=CLrAvVIOZ5t},
}
Figure