Abstract
To fight against real-life image forgery, which commonly involves different types and combined manipulations, we propose a unified deep neural architecture called ManTra-Net. Unlike many existing solutions, ManTra-Net is an end-to-end network that performs both detection and localization without extra preprocessing and postprocessing. ManTra-Net is a fully convolutional network and handles images of arbitrary sizes and many known forgery types such as splicing, copy-move, removal, enhancement, and even unknown types. We design a simple yet effective self-supervised learning task to learn robust image manipulation traces from classifying 385 image manipulation types. Further, we formulate the forgery localization problem as a local anomaly detection problem, design a Z-score feature to capture local anomaly, and propose a novel long short-term memory solution to assess local anomalies. Our extensive experimental results demonstrate the generalizability, robustness and superiority of ManTra-Net.
Introduction
Image forgery has become a digital epidemic, fueling everything from fake news to insurance fraud. While we often think of "Photoshopping" as a single act, real-world forgeries are complex. They involve splicing (pasting from another image), copy-move (cloning parts of the same image), removal (inpainting over an object), and enhancements (sharpening or adjusting brightness to make the edit blend in).
Existing forensic tools usually suffer from three major flaws: specialization (designed for only one type of forgery), lack of localization (can tell if an image is forged, but not where), and complexity (rely on slow, hand-crafted preprocessing). ManTra-Net addresses these by treating forgery as a statistical "odd-one-out" problem.
Methodology
ManTra-Net consists of two primary sub-networks:
- Image Manipulation Trace Feature Extractor: Instead of training on a few types of edits, the model uses a self-supervised task: classifying 385 different image manipulation types. This forces the VGG-style backbone to learn a rich library of "traces" left behind by various software operations.
- Local Anomaly Detection Network (LADN): This sub-network looks for anomalies. If a small portion of an image has different statistical traces than the rest, it is flagged.
- Z-score Features: The network calculates a Z-score for local features, standardizing how much a local patch deviates from the rest of the image.
- ConvLSTM2D: The model uses a Convolutional LSTM to analyze these Z-scores across multiple window sizes in a "far-to-near" sequential analysis, mimicking human inspection.
Experimental Evaluation
The model was trained using the Dresden Image Database to synthesize a massive 385-class manipulation set. Evaluation was conducted on standard benchmarks like NIST 2016, Columbia, COVERAGE, and CASIA, as well as the "PS-battle" dataset from Reddit, representing diverse, high-quality, real-life forgeries.
Key Results
ManTra-Net achieved state-of-the-art performance on COVERAGE and CASIA, outperforming previous deep learning methods like RGB-N and J-LSTM. It is remarkably resilient to "attacks" like blurring or noise addition. Efficiency is a major advantage; it can process a $1024 \times 768$ image in approximately 0.8 seconds on a single GPU.
Discussion and Conclusion
The core innovation of ManTra-Net is viewing forgeries as "statistical intruders." By focusing on local inconsistencies rather than specific forgery "looks," the model can detect types of manipulations it has never even seen before.
However, limitations remain. If an image is entirely synthesized (e.g., via GANs), there is no "pristine" background for comparison, causing anomaly detection to fail. Future work involves expanding the manipulation task to include modern types like DeepFakes and improving the system's ability to handle multiple disjoint forged regions.
Summary updated