SPAN: Spatial Pyramid Attention Network forImage Manipulation Localization

Xuefeng Hu, Zhihan Zhang, Zhenye Jiang, Syomantak Chaudhuri, Zhenheng Yang, Ram Nevatia

Abstract

We present a novel framework, Spatial Pyramid Attention Network (SPAN) for detection and localization of multiple types of image manipulations. The proposed architecture efficiently and effectively models the relationship between image patches at multiple scales by constructing a pyramid of local self-attention blocks. The design includes a novel position projection to encode the spatial positions of the patches. SPAN is trained on a generic, synthetic dataset but can also be fine-tuned for specific datasets; the proposed method shows significant gains in performance on standard datasets over previous state-of-the-art methods.

Introduction

As image editing tools become more sophisticated, creating "authentic-looking" forgeries—such as splicing an object from one photo into another or performing copy-move operations—has become trivial. The primary challenge for forensic researchers is localization: identifying exactly which pixels in an image have been tampered with.

Existing methods often suffer from two major limitations. First, they are frequently specialized for a single type of forgery (e.g., copy-move only). Second, general-purpose models often fail to capture the spatial relationships between different parts of an image. While some models analyze vertical relationships across scales, they often ignore horizontal relationships between neighboring pixels. SPAN aims to provide a general-purpose, multi-scale solution that understands both local artifacts and the broader image context.

Proposed Architecture

The SPAN architecture consists of three main components designed to capture multi-scale dependencies efficiently.

The SPAN architecture overview, showing the feature extraction, hierarchical local self-attention propagation, and decision modules.

1. Feature Extractor

The model utilizes a pre-trained backbone (derived from ManTra-Net) that employs VGG-style layers combined with specialized Bayer and SRM (Steganalysis Rich Model) filters. This allows the network to extract both visual artifacts and hidden noise patterns indicative of manipulation.

2. Spatial Pyramid Attention (SPA) Block

This is the core innovation of SPAN. Instead of computationally expensive global self-attention, SPAN uses Local Self-Attention (LSA). These blocks are stacked in a pyramid with increasing dilation distances (1, 3, 9, 27, 81). At the lowest layer, a pixel observes its immediate neighbors; at the highest layer, the pyramid structure allows a single pixel to effectively represent information from a 243x243 area. This approach achieves a massive receptive field while maintaining manageable computational complexity.

3. Position Projection and Decision

Unlike standard Transformers that use additive positional embeddings, SPAN introduces "Position Projection." These are learnable matrix projections that explicitly encode the relative spatial relationship between a neighbor pixel and a query pixel, allowing the model to better understand the geometry of tampered regions. Finally, a decision module consisting of 2D convolutional layers processes these high-level attention features to output a pixel-level mask.

Experimental Results

SPAN was pre-trained on a synthetic dataset containing 385 types of manipulations and evaluated on four classic benchmarks: NIST16, Columbia, Coverage, and CASIA (v1 and v2).

Quantitative Performance

SPAN demonstrated significant performance gains over previous state-of-the-art models like ManTra-Net and RGB-N. On the Columbia dataset, it achieved an 11.21% improvement in AUC over ManTra-Net without any fine-tuning. On the Coverage dataset, it showed a 9.5% improvement over RGB-N after fine-tuning. Furthermore, the model proved highly robust to JPEG compression, maintaining performance even at a quality factor of 50.

Qualitative Analysis

Visually, SPAN excels at identifying the interior of tampered regions, whereas previous models often only detected edges or produced fragmented masks.

Qualitative results showing manipulated images, SPAN's predicted masks, and the ground truth (GT) masks across various forgery types.

Conclusion

SPAN introduces an efficient way to model spatial relationships at multiple scales for image forensics. By leveraging dilated local self-attention and a novel position projection method, it provides a robust and generalizable framework for manipulation localization. Future work will explore making the network more resilient to aggressive resizing and extending these mechanisms to video manipulation detection.

Summary updated

Abstract

We present a novel framework, Spatial Pyramid Attention Network (SPAN) for detection and localization of multiple types of image manipulations. The proposed architecture efficiently and effectively models the relationship between image patches at multiple scales by constructing a pyramid of local self-attention blocks. The design includes a novel position projection to encode the spatial positions of the patches. SPAN is trained on a generic, synthetic dataset but can also be fine tuned for specific datasets; The proposed method shows significant gains in performance on standard datasets over previous state-of-the-art methods.
BibTeX
@inproceedings{hu2020span,
  author = {Xuefeng Hu and Zhihan Zhang and Zhenye Jiang and Syomantak Chaudhuri and Zhenheng Yang and Ram Nevatia},
  title = {{SPAN: Spatial Pyramid Attention Network forImage Manipulation Localization}},
  booktitle = {European Conference on Computer Vision},
  year = {2020},
  url = {https://openreview.net/forum?id=52A2Z_C6Wwm},
}
Figure