TL;DR
This paper introduces an attention-based dual-branch convolutional neural network (AD-CNN) for detecting computer-generated graphics (CGs) in images, particularly excelling with small image patches and post-processed images. By fusing raw and blurred RGB components and employing an attention mechanism to weigh features from multi-scale branches, the AD-CNN achieves superior robustness and accuracy compared to existing methods.
What Problem Does This Paper Address?
The increasing realism of computer graphics (CGs) makes it harder to distinguish them from real photographic images (PGs). This poses a significant threat to the authenticity and integrity of digital content, with implications for fields like journalism, forensics, and academic research. While deep learning methods have shown promise in CG detection, existing approaches struggle with small image patches and common post-processing operations (like resizing or noise addition). This is because such operations can degrade or remove the subtle forensic traces left by CG generation processes, making detection unreliable. The paper aims to address this gap by developing a more robust and accurate CG detection method that can handle these challenging scenarios.
Methods
The proposed method, AD-CNN, employs a novel architecture and preprocessing strategy.
- Preprocessing (Early Fusion): Instead of using raw RGB components alone, the authors propose fusing them with their blurred versions. This is achieved by applying a Gaussian low-pass filter to the raw RGB components to generate "blurred RGB components." These raw and blurred components are then concatenated channel-wise, creating a 6-channel input. This "early fusion" strategy aims to provide the network with both detailed local information (from raw components) and more generalized, robust patterns (from blurred components), making it less susceptible to post-processing artifacts.
- Dual-Branch CNN: The core of the AD-CNN is a dual-branch convolutional neural network. Both branches share an identical shallow CNN architecture (4 convolutional modules each). However, the crucial difference lies in their first convolutional layer, where one branch uses a 3x3 kernel and the other uses a 5x5 kernel. This multi-scale approach allows each branch to capture different levels of low-level forensic traces.
- Attention-Based Fusion Module: The features extracted from the two branches are then fed into an attention-based fusion module. This module adaptively learns to assign different weights to the features from each branch, prioritizing the more informative ones for CG detection. This is achieved by computing a "significance" score for each branch's features and then using a softmax function to derive weights that sum to one. This allows the network to dynamically adjust its focus based on the input image.
- Classification: The fused features are then passed through three fully-connected layers for final classification, outputting the probability of the image patch being a CG.
Datasets
The study utilizes a publicly available dataset of CGs and PGs, previously introduced in [22]. This dataset contains 6800 CGs and 6800 PGs, with resolutions ranging from 266x199 to 2048x3200. For evaluation, the authors specifically extract small image patches of sizes 32x32 and 64x64 from the central regions of these images. The dataset is split into training and testing sets with a 4:1 ratio, and the training set is further divided for validation.
Key Insights and Ideas
- Blurring for Robustness: The simple act of blurring RGB components before feeding them into the network significantly improves robustness against post-processing. This is because low-frequency components are less sensitive to operations like resizing or noise addition compared to high-frequency components.
- Early Fusion is Key: Concatenating raw and blurred components at the input layer (early fusion) outperforms late fusion (where components are processed separately and then combined). This suggests that the network benefits more from learning correlations between raw and blurred information from the outset.
- Attention Matters: The attention-based fusion module is crucial for achieving the best performance. It allows the network to intelligently weigh features from different branches, recognizing that multi-scale features have varying importance for different image content.
- Multi-Scale Branches: The dual-branch structure with different kernel sizes in the initial layers is effective for capturing diverse low-level forensic traces that are characteristic of CG generation.
Key Results
The proposed AD-CNN achieved state-of-the-art performance on the tested dataset, particularly for small image patches:
- Small Patches (32x32): The AD-CNN achieved an accuracy of 83.75%, outperforming NoiseNet (75.53%), CGNet (80.32%), and ScNet (79.21%).
- Larger Patches (64x64): The AD-CNN achieved an accuracy of 87.82%, outperforming NoiseNet (79.19%), CGNet (82.01%), ScNet (82.63%), and HeNet (82.57%). The performance improvement over these methods ranged from 5.19% to 8.63%.
- Robustness to JPEG Compression: The AD-CNN demonstrated superior robustness to JPEG compression, showing a smaller performance drop compared to other methods. For a quality factor of 75, the AD-CNN achieved 81.88% accuracy, while ScNet dropped to 71.51%.
- Robustness to Other Post-Processing: The method also showed strong performance against various other post-processing operations like bilateral filtering, median filtering, resizing, sharpening, and noise addition, consistently outperforming existing methods.
Limitations and Future Work
- Localization: The current work focuses on detecting whether an image patch is CG or PG. The authors plan to extend their method to localize tampered regions created by CGs in future work.
- Computational Efficiency: While the paper focuses on accuracy and robustness, further optimization for computational efficiency might be explored.
- Dataset Size: The authors mention that the scale of training samples can be limited, which motivates their preprocessing strategies. Exploring larger and more diverse datasets could further enhance generalization.
Summary updated