ManTra-Net: Manipulation Tracing Network for Detection and Localization of Image Forgeries With Anomalous Features.

Yue Wu, Wael AbdAlmageed, Premkumar NatarajanCVPR 2019 · OpenReview

Abstract

To fight against real-life image forgery, which commonly involves different types and combined manipulations, we propose a unified deep neural architecture called ManTra-Net. Unlike many existing solutions, ManTra-Net is an end-to-end network that performs both detection and localization without extra preprocessing and postprocessing. ManTra-Net is a fully convolutional network and handles images of arbitrary sizes and many known forgery types such as splicing, copy-move, removal, enhancement, and even unknown types. We design a simple yet effective self-supervised learning task to learn robust image manipulation traces from classifying 385 image manipulation types. Further, we formulate the forgery localization problem as a local anomaly detection problem, design a Z-score feature to capture local anomaly, and propose a novel long short-term memory solution to assess local anomalies. Our extensive experimental results demonstrate the generalizability, robustness and superiority of ManTra-Net.

Introduction

Image forgery has become a digital epidemic, fueling everything from fake news to insurance fraud. While we often think of "Photoshopping" as a single act, real-world forgeries are complex. They involve splicing (pasting from another image), copy-move (cloning parts of the same image), removal (inpainting over an object), and enhancements (sharpening or adjusting brightness to make the edit blend in).

Existing forensic tools usually suffer from three major flaws: specialization (designed for only one type of forgery), lack of localization (can tell if an image is forged, but not where), and complexity (rely on slow, hand-crafted preprocessing). ManTra-Net addresses these by treating forgery as a statistical "odd-one-out" problem.

Methodology

ManTra-Net consists of two primary sub-networks:

  1. Image Manipulation Trace Feature Extractor: Instead of training on a few types of edits, the model uses a self-supervised task: classifying 385 different image manipulation types. This forces the VGG-style backbone to learn a rich library of "traces" left behind by various software operations.
  2. Local Anomaly Detection Network (LADN): This sub-network looks for anomalies. If a small portion of an image has different statistical traces than the rest, it is flagged.
  3. Z-score Features: The network calculates a Z-score for local features, standardizing how much a local patch deviates from the rest of the image.
  4. ConvLSTM2D: The model uses a Convolutional LSTM to analyze these Z-scores across multiple window sizes in a "far-to-near" sequential analysis, mimicking human inspection.

Experimental Evaluation

The model was trained using the Dresden Image Database to synthesize a massive 385-class manipulation set. Evaluation was conducted on standard benchmarks like NIST 2016, Columbia, COVERAGE, and CASIA, as well as the "PS-battle" dataset from Reddit, representing diverse, high-quality, real-life forgeries.

Key Results

ManTra-Net achieved state-of-the-art performance on COVERAGE and CASIA, outperforming previous deep learning methods like RGB-N and J-LSTM. It is remarkably resilient to "attacks" like blurring or noise addition. Efficiency is a major advantage; it can process a $1024 \times 768$ image in approximately 0.8 seconds on a single GPU.

Example of a forged sample created by splicing elements from different source images.

Discussion and Conclusion

The core innovation of ManTra-Net is viewing forgeries as "statistical intruders." By focusing on local inconsistencies rather than specific forgery "looks," the model can detect types of manipulations it has never even seen before.

However, limitations remain. If an image is entirely synthesized (e.g., via GANs), there is no "pristine" background for comparison, causing anomaly detection to fail. Future work involves expanding the manipulation task to include modern types like DeepFakes and improving the system's ability to handle multiple disjoint forged regions.

Summary updated

Abstract

To fight against real-life image forgery, which commonly involves different types and combined manipulations, we propose a unified deep neural architecture called ManTra-Net. Unlike many existing solutions, ManTra-Net is an end-to-end network that performs both detection and localization without extra preprocessing and postprocessing. \manifold is a fully convolutional network and handles images of arbitrary sizes and many known forgery types such splicing, copy-move, removal, enhancement, and even unknown types. This paper has three salient contributions. We design a simple yet effective self-supervised learning task to learn robust image manipulation traces from classifying 385 image manipulation types. Further, we formulate the forgery localization problem as a local anomaly detection problem, design a Z-score feature to capture local anomaly, and propose a novel long short-term memory solution to assess local anomalies. Finally, we carefully conduct ablation experiments to systematically optimize the proposed network design. Our extensive experimental results demonstrate the generalizability, robustness and superiority of ManTra-Net, not only in single types of manipulations/forgeries, but also in their complicated combinations.
BibTeX
@inproceedings{wu2019mantra,
  author = {Yue Wu and Wael AbdAlmageed and Premkumar Natarajan},
  title = {{ManTra-Net: Manipulation Tracing Network for Detection and Localization of Image Forgeries With Anomalous Features.}},
  booktitle = {CVPR},
  year = {2019},
  url = {https://openreview.net/forum?id=HXKVVCflOpB},
}
Figure