Abstract
While current deepfake detection methods excel at identifying synthetic content and general architectures, they often fail to provide specific model attribution. This paper presents a novel approach to "image ballistics" for deepfakes, identifying the unique set of weights (model instance) that generated an image. By exploiting subtle fingerprints in the latent space of GANs and utilizing metric learning on early neural network layers, the proposed method achieves over 96% accuracy in attributing images to specific models, even when those models share identical architectures.
Introduction
The rapid advancement of Generative Adversarial Networks (GANs) has made the creation of hyper-realistic synthetic media trivial. Current state-of-the-art detectors are proficient at spotting synthetic images and identifying broad architectures (e.g., StyleGAN vs. CycleGAN). However, a critical gap remains in model attribution: identifying which specific instance of an architecture—defined by its unique training run and weights—produced the image. This level of granularity is essential for protecting Intellectual Property (IP) and establishing accountability for malicious content dissemination.
Methods and Pipeline
The authors propose a robust processing pipeline consisting of three main stages to bridge the gap between detection and attribution:
- Model Creation: 100 unique versions of the StyleGAN2-ADA architecture were generated by varying fine-tuning hyperparameters, such as training-set augmentation probability and the number of images processed.
- Encoder Training (Baseline): A ResNet-18 encoder was trained to classify images among 50 "known" models. The authors discovered that the first layer (Layer-1) of the ResNet was the most effective for feature extraction, as it captures subtle, low-level "fingerprints" rather than high-level semantic content.
- Metric Learning (Generalization): To handle "unknown" models, the authors implemented a metric learning approach using Singular Value Decomposition (SVD) for dimensionality reduction and a k-Nearest Neighbors (k-NN) classifier to determine image provenance.
Key Insights: Semantics vs. Forensics
A pivotal finding of this research is the distinction between semantic understanding and forensic analysis. While standard AI tasks aim to ignore noise to understand content, forensics seeks to ignore content to find the "noise" or artifacts left by the generator.
As shown in the feature map visualizations, the earliest layers of the neural network are the most useful for recognition because they retain the high-frequency signatures unique to a specific GAN instance. Even slight changes in training parameters, like the augmentation probability $p$, are sufficient to create a distinct, recognizable model fingerprint.
Experimental Results
- Known Model Classification: The ResNet-18 encoder achieved a classification accuracy of 96.24% across 50 known models.
- Unknown Model Recognition: The metric learning approach achieved 94.33% accuracy (5-NN) in identifying images from models never seen during the training phase.
- Robustness: The system maintained 81.37% accuracy against common image attacks, including Gaussian blur, JPEG compression, and scaling.
- Cross-Architecture Scalability: When tested against StyleGAN1, the metric maintained near 100% distance separation, ensuring the system does not confuse different architectures with specific model instances.
Conclusion and Future Work
This work demonstrates that GAN model instances leave exploitable fingerprints that allow for high-accuracy attribution. While the method is robust to moderate noise, extreme compression (e.g., social media re-encoding) remains a challenge, dropping accuracy to approximately 80%. Future research will focus on the scalability of these fingerprints as the number of potential source models grows and testing the methodology against other architectures like Diffusion Models and VAEs.
Summary updated