Splicing Detection and Localization In Satellite Imagery Using Conditional GANs

Emily R. Bartusiak, Sri Kalyan Yarlagadda, David Güera, Paolo Bestagini, Stefano Tubaro, Fengqing M. Zhu, Edward J. Delp

TL;DR

This paper introduces a deep learning approach using Conditional Generative Adversarial Networks (cGANs) to detect and precisely locate spliced forgeries in satellite imagery. By treating forgery localization as an image-to-mask translation task and optimizing with binary cross-entropy loss, the researchers achieved near-perfect detection and high-fidelity localization, even for forgery sizes the model never saw during training.

What Problem Does This Paper Address?

In an era where sophisticated image editing tools like Photoshop and GIMP are ubiquitous, the integrity of digital media is constantly under threat. While much of the digital forensics field focuses on images from consumer cameras and smartphones, satellite imagery presents a unique and critical challenge. These images are vital for government intelligence, environmental monitoring, and business logistics, yet they are susceptible to "splicing"—the insertion or removal of objects (like airplanes or clouds) to hide or fabricate scenes.

The authors identify a gap: satellite images have different acquisition characteristics than standard photos, and existing forensic tools often struggle with them. They aim to solve two specific problems: detection (determining if an image has been tampered with) and localization (identifying exactly which pixels belong to the forged object).

Methods

The researchers employ a Conditional Generative Adversarial Network (cGAN), specifically an extension of the popular pix2pix architecture. The system consists of two competing networks:

  1. The Generator ($G$): A 16-layer U-Net that takes a satellite image as input and attempts to produce a "forgery mask." In this mask, black pixels represent pristine areas, and white pixels represent the detected forgery.
  2. The Discriminator ($D$): A 5-layer CNN that looks at pairs of images and masks. Its job is to distinguish between "real" ground-truth masks and the "fake" masks produced by the generator.
The cGAN architecture showing how the Generator and Discriminator interact during training.

A key technical contribution is the choice of loss function. While the original pix2pix uses $L_1$ loss for reconstruction, these authors argue that because their goal is essentially pixel-wise binary classification (forged vs. pristine), Binary Cross-Entropy (BCE) loss is a superior choice. This forces the generator to produce sharper, more decisive masks.

Datasets

The study utilized a dataset derived from NASA/USGS Landsat imagery. The dataset consists of $650 \times 650$ RGB images where objects like airplanes and clouds were spliced in at random locations.

The data is categorized by forgery size:

  • Small: ~32x32 pixels
  • Medium: ~64x64 pixels
  • Large: ~128x128 pixels

Crucially, the model was trained and validated only on pristine and small forgery samples. The medium and large forgeries were reserved strictly for testing to evaluate how well the model generalizes to unseen scales.

Key Insights and Ideas

  • Direct Mapping vs. One-Class Learning: Previous work often tried to learn a "compact representation" of pristine data to identify forgeries as outliers. This paper shows that providing examples of both pristine and forged data during training allows the model to learn a more robust direct mapping to forgery masks.
  • The Power of BCE Loss: The transition from $L_1$ to BCE loss was pivotal. Because the target output is binary (0 or 255), BCE aligns better with the underlying statistics of the task, leading to significantly higher localization accuracy.
  • Scale Generalization: One of the most impressive "aha moments" is that a model trained only on small forgeries can accurately identify much larger ones. This suggests the cGAN is learning the local artifacts of splicing (like edge inconsistencies or statistical mismatches) rather than just the shapes of the objects themselves.
Examples of input images, ground truth masks, and the masks generated by the model across different forgery sizes.

Key Results

The results demonstrate that the cGAN approach is highly effective:

  • Detection: The model achieved an Area Under the Curve (AUC) and Average Precision of 1.000, meaning it perfectly distinguished between pristine and forged images in the test set.
  • Localization: When using the preferred BCE loss, the model achieved an AUC of 0.988 and an Average Precision of 0.953.
  • Comparison: The BCE loss significantly outperformed the standard $L_1$ loss for localization (which had an AUC of 0.927), validating the authors' architectural choice.

Limitations and Future Work

While the results are excellent, the authors acknowledge a few areas for further exploration:

  • Forgery Types: This study focused specifically on splicing. Future work should test the model against other manipulations like inpainting or copy-move forgeries.
  • Sensor Diversity: The dataset was limited to Landsat imagery. Testing on data from various satellites with different resolutions and spectral characteristics is necessary to ensure the method is truly "sensor-agnostic."
  • Real-world Complexity: The spliced objects (planes/clouds) are relatively distinct. Testing on more subtle, context-aware forgeries would be a rigorous next step.

Summary updated

Abstract

The widespread availability of image editing tools and improvements in image processing techniques allow image manipulation to be very easy. Oftentimes, easy-to-use yet sophisticated image manipulation tools yields distortions/changes imperceptible to the human observer. Distribution of forged images can have drastic ramifications, especially when coupled with the speed and vastness of the Internet. Therefore, verifying image integrity poses an immense and important challenge to the digital forensic community. Satellite images specifically can be modified in a number of ways, including the insertion of objects to hide existing scenes and structures. In this paper, we describe the use of a Conditional Generative Adversarial Network (cGAN) to identify the presence of such spliced forgeries within satellite images. Additionally, we identify their locations and shapes. Trained on pristine and falsified images, our method achieves high success on these detection and localization objectives.
BibTeX
@inproceedings{bartusiak2019splicing,
  author = {Emily R. Bartusiak and Sri Kalyan Yarlagadda and David Güera and Paolo Bestagini and Stefano Tubaro and Fengqing M. Zhu and Edward J. Delp},
  title = {{Splicing Detection and Localization In Satellite Imagery Using Conditional GANs}},
  booktitle = {MIPR},
  year = {2019},
}
Figure