CAT-Net: Compression Artifact Tracing Network for Detection and Localization of Image Splicing

Myung-Joon Kwon, In-Jae Yu, Seung-Hun Nam, Heung-Kyu LeeWACV 2021

TL;DR

This paper introduces CAT-Net, a novel deep learning architecture that simultaneously analyzes both visual (RGB) and compression artifact (DCT) features to detect and precisely locate spliced regions in images. By jointly leveraging these complementary domains, CAT-Net achieves state-of-the-art performance, outperforming existing methods, especially in challenging JPEG compressed images.

What Problem Does This Paper Address?

The proliferation of user-friendly image editing tools and social media has made image manipulation, particularly splicing (copy-pasting parts of one image into another), a significant threat. Forged images can be used to spread misinformation, create fake evidence, or propagate propaganda. Detecting and accurately localizing these spliced regions is crucial for maintaining trust in digital media. A major challenge lies in distinguishing between authentic and tampered areas, especially when the forgery is subtle and the image is compressed using formats like JPEG, which introduce artifacts that can obscure or mimic manipulation traces. Existing methods often struggle to effectively utilize these compression artifacts, particularly when dealing with varying compression levels or when the forgery itself is designed to blend in.

Methods

CAT-Net employs an end-to-end fully convolutional neural network architecture that ingeniously combines two distinct streams: an RGB stream and a DCT stream.

  • RGB Stream: This stream processes the raw pixel values of the image, learning to identify visual clues and anomalies that might indicate tampering. It utilizes a modified HRNet backbone, known for its ability to maintain high-resolution representations throughout the network, which is beneficial for capturing fine-grained forensic details.
  • DCT Stream: This stream focuses on compression artifacts. It takes quantized Y-channel DCT coefficients and the corresponding quantization table as input. A key innovation here is the "JPEG artifact learning module." This module converts DCT coefficients into a binary volume representation, allowing the network to learn relationships between adjacent coefficients. It also incorporates a mechanism to utilize the quantization table, which is crucial for understanding how the image was compressed. This stream is pre-trained on a double JPEG detection task, equipping it with a strong understanding of compression-induced artifacts.
  • Fusion Stage: The outputs from both the RGB and DCT streams are fused in a multi-resolution fusion stage. This allows the network to combine the strengths of both streams, leveraging visual cues from the RGB stream and compression artifact traces from the DCT stream to generate a precise localization mask.

Datasets

The paper utilizes several publicly available datasets for training and evaluation:

  • Training: CASIA v2, Fantastic Reality, IMD2020, and a newly created "Spliced COCO" dataset. The Spliced COCO dataset was generated by automatically splicing objects from COCO images and compressing them with random JPEG quality factors to create a more diverse benchmark.
  • Testing: CASIA v2, Fantastic Reality, IMD2020, Spliced COCO, NC16 Splicing, Carvalho, and Columbia. Notably, the Columbia dataset is mentioned as violating the compression assumption, making it a challenging test case.

Key Insights and Ideas

  • Joint RGB and DCT Analysis is Crucial: The paper strongly demonstrates that relying solely on visual (RGB) information is insufficient for robust splicing detection, especially in compressed images. The DCT stream's ability to trace compression artifacts provides complementary and vital clues.
  • Pre-training on Double JPEG Detection is Effective: Initializing the DCT stream with weights from a double JPEG detection task significantly enhances its ability to capture compression artifacts, which directly benefits the splicing localization task. This transfer learning approach is a novel application in this domain.
  • Binary Volume Representation for DCT Coefficients: Moving beyond traditional DCT histograms, the binary volume representation allows for pixel-wise predictions and better captures spatial relationships within DCT coefficients, leading to improved performance.
  • Quantization Table Matters: The inclusion of the quantization table in the DCT stream's processing significantly improves performance, highlighting its importance in understanding the compression history of an image.
  • HRNet Backbone for High-Resolution Features: The use of HRNet as a backbone for both streams helps maintain high-resolution feature maps, which are essential for precise localization of subtle manipulation traces.

Key Results

CAT-Net achieved state-of-the-art performance across multiple benchmark datasets.

  • Overall Performance: In Table 3, CAT-Net consistently outperforms ManTra-Net and EXIF consistency across various datasets, achieving high accuracy (e.g., 99.66% Acc on CASIA v2 authentic) and mIoU (e.g., 87.69% p-mIoU on Fantastic Reality tampered).
  • Robustness to JPEG Compression: Figure 7 shows that CAT-Net exhibits superior robustness to varying JPEG quality factors compared to other methods, maintaining good performance even at lower quality settings.
  • Ablation Studies: Table 3 also shows that while the RGB stream alone performs well, and the DCT stream alone shows promise, their combination in CAT-Net leads to the best overall performance, demonstrating their complementary nature. For instance, in Carvalho (tampered), the DCT stream performed better, while in NC16 Splicing (tampered), the RGB stream was stronger, but the full CAT-Net excelled in both.

Limitations and Future Work

  • Columbia Dataset Challenge: The paper acknowledges that the Columbia dataset's lack of compression artifacts poses a challenge for the DCT stream. While CAT-Net still performs well due to the RGB stream, this highlights a limitation when dealing with images that have undergone manipulation without prior compression.
  • Non-JPEG Image Handling: While the paper describes a method for handling non-JPEG images by converting them to JPEG with quality 100, this might not perfectly replicate the original image characteristics and could potentially introduce artifacts.
  • Computational Cost: Although the paper mentions fast inference, deep learning models, especially those with multiple streams and high-resolution processing, can still be computationally intensive.

Future work could explore more sophisticated methods for handling non-JPEG images or investigate architectures that are inherently more robust to the absence of compression artifacts. Further research into fusing features from even more diverse domains could also lead to even more powerful forgery detection systems.

Summary updated

BibTeX
@inproceedings{kwon2021cat,
  author = {Myung-Joon Kwon and In-Jae Yu and Seung-Hun Nam and Heung-Kyu Lee},
  title = {{CAT-Net: Compression Artifact Tracing Network for Detection and Localization of Image Splicing}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2021},
}
Figure