TL;DR
The authors introduce WILD (in-the-Wild Image Linkage Dataset), a new benchmark designed to train and evaluate synthetic image source attribution models under realistic conditions. By incorporating 20 different generative models divided into "closed" and "open" sets, alongside multi-step post-processing pipelines, the dataset reveals a critical gap in current multimedia forensics: traditional CNNs excel in pristine laboratory settings but fail catastrophically when images are edited or compressed, whereas Vision Transformers (ViTs) offer much greater real-world robustness.
What Problem Does This Paper Address?
With the explosion of Generative AI, distinguishing real images from fakes is no longer the only challenge; forensic investigators also need to know which specific generator produced a synthetic image. This task—known as synthetic image source attribution or image linkage—is crucial for intellectual property protection, tracking misinformation campaigns, and legal accountability.
However, existing datasets suffer from major limitations: they often feature low-resolution images, lack commercial models (like Midjourney or Adobe Firefly), ignore real-world image editing (like social media compression or resizing), and fail to include an "open set" of unknown generators to simulate unpredictable real-world scenarios. The authors introduce WILD to bridge this gap.
Methods
To construct a bias-free and realistic dataset, the authors developed an automated prompt generation pipeline. They utilized five distinct sentence structures and randomized key attributes (gender, age, profession, facial features, clothing, background, and looking direction) to generate 1,000 unique prompts.
For the Closed Set, every single prompt was fed into 10 popular text-to-image generators, ensuring a perfectly uniform data distribution across classes and creating a direct link between each image and its prompt.
To simulate real-world image degradation (e.g., sharing via social media), half of the images underwent a randomized post-processing pipeline consisting of 1 to 3 sequential operations chosen from ten common transformations, including JPEG/WebP compression, Gaussian blur, resizing, rotation, and super-resolution.
The authors then benchmarked seven state-of-the-art baseline methodologies on WILD:
- CLIP Feature Classifiers (paired with an MLP or SVM classification head)
- DE-FAKE: A multimodal framework utilizing both image and text tokens
- Standard CNNs: EfficientNetB4, XceptionNet, and ResNet50 trained at the patch level
- Vision Transformer Classifier (VTC): A ViT-Base model fine-tuned directly for attribution
Datasets
The paper introduces the WILD dataset, which consists of:
- Closed Set (10,000 images): 1,000 images each from 10 mainstream commercial and open-source models: Adobe Firefly (ADF), DALL·E 3 (DE3), FLUX.1 (FX1), FLUX 1.1 Pro (FXP), Freepik (FPK), Leonardo AI (LEO), Midjourney (MDJ), Stable Diffusion 3.5 Large (S35), Stable Diffusion XL Turbo (SXL), and Starry AI (STR).
- Open Set (10,000 images): 1,000 images each from 10 alternative models to simulate unseen "in-the-wild" generators, including older models, GANs, and alternative architectures: DALL·E 1 (DE1), Deep AI (DPA), Hotpot AI (HPA), NVIDIA Sana PAG (NVS), Stable Cascade (SCD), Stable Diffusion Attend&Excite (SAE), StyleGAN (SG1), StyleGAN2 (SG2), StyleGAN3 (SG3), and Tencent Hunyuan (THY).
Combined with post-processed variants, the dataset contains a grand total of 50,000 images, focusing exclusively on head-and-shoulder portraits of synthetic people.
Key Insights and Ideas
- The Artifact Fragility Dilemma: CNN-based models achieve near-perfect accuracy (100%) on pristine, unedited images because they latch onto low-level hardware and software artifacts (the "fingerprints" of the generator). However, common editing operations like blurring or compression wipe away these micro-traces, causing CNN performance to plummet.
- Semantics Over Signatures: Vision Transformers (VTC) prove to be significantly more robust against post-processing. Because ViTs leverage self-attention to capture long-range dependencies and higher-level semantic structures, they maintain high accuracy even when low-level artifacts are destroyed.
- The Open-Set Challenge: Distinguishing known models from entirely unknown generators (open-set rejection) remains incredibly difficult. While some models can successfully reject unknown models under ideal conditions, their ability to do so degrades rapidly when any form of image editing is introduced.
Key Results
- Closed-Set Attribution: On plain images, CNNs (EfficientNetB4, XceptionNet) achieved 100% balanced accuracy. However, after 3 steps of post-processing, their accuracy dropped sharply to ~59%. Conversely, the Vision Transformer Classifier (VTC) started at 95.80% on plain images but maintained an impressive 88.67% accuracy after 3 steps of editing.
- Open-Set Rejection: In a multi-class setup with a rejection threshold (aiming for a 5% false open-set detection rate on validation data), XceptionNet achieved a perfect Correct Classification Rate (CCR) of 1.0 across all known classes and a Correct Rejection Rate (CRR) of 0.70 for unknown open-set models on plain images.
Limitations and Future Work
The primary limitation of the WILD dataset is its narrow semantic scope: it consists exclusively of head-and-shoulder portraits of synthetic human faces generated from fixed prompt structures. While this controls for content bias, real-world forensic scenarios involve diverse objects, landscapes, and text.
The authors suggest that future work should expand the dataset's diversity and explore how prompt semantics explicitly alter the artifacts left by different generators. Additionally, they highlight the need for explainability studies to better understand exactly what features ViT-based models rely on when performing robust source attribution.
Summary updated