TL;DR
This paper pioneers the field of "deepfake ballistics" by demonstrating that we can detect not just if an image is a deepfake, but how many times it has been processed by a style-transfer generative network. By analyzing frequency domain statistics and leveraging deep learning, the authors achieve up to 92.75% accuracy in distinguishing single-manipulated from double-manipulated synthetic faces, while also uncovering unique mathematical properties of the style-transfer operation itself.
What Problem Does This Paper Address?
Multimedia forensics has become highly proficient at distinguishing real images from deepfakes and identifying the specific Generative Adversarial Network (GAN) architecture used to create them. However, a major gap remains: reconstructing the history or lineage of a synthetic image—a field known as image ballistics.
If a malicious actor applies a style-transfer algorithm to a face once, and then applies another layer of manipulation later, can we detect this multi-stage processing? The authors address this challenge by investigating whether generative manipulations leave cumulative, detectable traces, treating the problem similarly to double-quantization detection in traditional JPEG forensics.
Methods
The authors propose two distinct approaches to classify whether an image has undergone a single style-transfer operation (termed Deepfake-2) or a double operation (Deepfake-3):
- Analytical Approach (Method 1): Following standard JPEG processing pipelines, images are split into $8 \times 8$ non-overlapping blocks, and the Discrete Cosine Transform (DCT) is applied. While the DC coefficient represents the average pixel value, the 63 AC coefficients are modeled using a zero-centered Laplacian distribution. The scale parameters ($\beta$) of these distributions form a 63-dimensional feature vector used to train traditional machine learning classifiers (e.g., Random Forest, SVM, k-NN).
- Deep Learning Approach (Method 2): A standard ResNet-18 architecture pre-trained on ImageNet is fine-tuned. The final fully-connected layer is modified to output a two-class distribution (single vs. double manipulation) followed by a SoftMax layer, optimized using Cross-Entropy loss.
Additionally, the paper formally tests the algebraic properties of the style-transfer operator ($\oplus$), evaluating Identity, Commutativity, and Associativity using Structural Similarity (SSIM) and RGB color histograms.
Datasets
The study utilizes images generated by StarGAN-V2, selected for its state-of-the-art ability to produce high-resolution, high-quality facial attribute transfers.
- Deepfake-2 (Single Pass): 1,200 training and 200 testing images generated by transferring attributes between two distinct source datasets.
- Deepfake-3 (Double Pass): 1,200 training and 200 testing images where the output of the first style-transfer pass is subjected to a second pass with a different target attribute.
In total, the benchmark dataset consists of 2,400 training images and 400 test images.
Key Insights and Ideas
- Cumulative Fingerprints: Even when the exact same GAN architecture (StarGAN-V2) is used for both passes, the second operation does not erase the first trace. Instead, it creates a unique "combination" fingerprint that remains distinct from a single-pass image.
- Asymmetry and Lack of Identity: Mathematically, style transfer behaves very differently from standard image operations. The authors prove that there is no true "neutral element" (Identity property fails)—even passing the exact same face as both source and target causes the GAN to reconstruct the image with subtle architectural artifacts.
- Quasi-Associativity: While style transfer is strictly non-commutative ($A \oplus B \neq B \oplus A$), it exhibits quasi-associativity when looking at color distributions. The order of operations in a multi-stage pipeline, $((A \oplus B) \oplus C)$, yields an RGB histogram nearly identical to $(A \oplus (B \oplus C))$, providing a strong hint for future forensic reverse-engineering.
Key Results
- ResNet-18 Performance: The deep learning approach significantly outperformed the analytical method, achieving a top classification accuracy of 92.75% with a loss value of 0.23.
- Analytical Performance: Among the traditional classifiers trained on DCT $\beta$ coefficients, the Random Forest model performed best, reaching 81% accuracy. Other classifiers like k-NN and linear SVM hovered around 73–75% accuracy.
Limitations and Future Work
- Architecture Specificity: The current study evaluates its methods exclusively on images generated by StarGAN-V2. It remains to be seen how well these forensic traces generalize across different generative architectures or when multiple different networks are mixed sequentially.
- Manipulation Types: The focus was strictly limited to the style-transfer category of attribute manipulation. The authors note that future work must expand these ballistics techniques to other common deepfake categories, such as face reenactment (expression swap) and face swapping, as well as extending the analysis to audio and video deepfakes.
Summary updated