WILD: a new in-the-Wild Image Linkage Dataset for synthetic image attribution

Pietro Bongini, Sara Mandelli, Andrea Montibeller, Mirko Casu, Orazio Pontorno, Claudio Vittorio Ragaglia, Luca Zanchetta, Mattia Aquilina, Taiba Majid Wani, Luca Guarnera, Benedetta Tondi, Giulia Boato, Paolo Bestagini, Irene Amerini, Francesco De Natale, Sebastiano Battiato, Mauro BarniarXiv 2025 · arXiv

TL;DR

The authors introduce WILD (in-the-Wild Image Linkage Dataset), a new benchmark designed to train and evaluate synthetic image source attribution models under realistic conditions. By incorporating 20 different generative models divided into "closed" and "open" sets, alongside multi-step post-processing pipelines, the dataset reveals a critical gap in current multimedia forensics: traditional CNNs excel in pristine laboratory settings but fail catastrophically when images are edited or compressed, whereas Vision Transformers (ViTs) offer much greater real-world robustness.

What Problem Does This Paper Address?

With the explosion of Generative AI, distinguishing real images from fakes is no longer the only challenge; forensic investigators also need to know which specific generator produced a synthetic image. This task—known as synthetic image source attribution or image linkage—is crucial for intellectual property protection, tracking misinformation campaigns, and legal accountability.

However, existing datasets suffer from major limitations: they often feature low-resolution images, lack commercial models (like Midjourney or Adobe Firefly), ignore real-world image editing (like social media compression or resizing), and fail to include an "open set" of unknown generators to simulate unpredictable real-world scenarios. The authors introduce WILD to bridge this gap.

Methods

To construct a bias-free and realistic dataset, the authors developed an automated prompt generation pipeline. They utilized five distinct sentence structures and randomized key attributes (gender, age, profession, facial features, clothing, background, and looking direction) to generate 1,000 unique prompts.

For the Closed Set, every single prompt was fed into 10 popular text-to-image generators, ensuring a perfectly uniform data distribution across classes and creating a direct link between each image and its prompt.

To simulate real-world image degradation (e.g., sharing via social media), half of the images underwent a randomized post-processing pipeline consisting of 1 to 3 sequential operations chosen from ten common transformations, including JPEG/WebP compression, Gaussian blur, resizing, rotation, and super-resolution.

The authors then benchmarked seven state-of-the-art baseline methodologies on WILD:

  1. CLIP Feature Classifiers (paired with an MLP or SVM classification head)
  2. DE-FAKE: A multimodal framework utilizing both image and text tokens
  3. Standard CNNs: EfficientNetB4, XceptionNet, and ResNet50 trained at the patch level
  4. Vision Transformer Classifier (VTC): A ViT-Base model fine-tuned directly for attribution

Datasets

The paper introduces the WILD dataset, which consists of:

  • Closed Set (10,000 images): 1,000 images each from 10 mainstream commercial and open-source models: Adobe Firefly (ADF), DALL·E 3 (DE3), FLUX.1 (FX1), FLUX 1.1 Pro (FXP), Freepik (FPK), Leonardo AI (LEO), Midjourney (MDJ), Stable Diffusion 3.5 Large (S35), Stable Diffusion XL Turbo (SXL), and Starry AI (STR).
  • Open Set (10,000 images): 1,000 images each from 10 alternative models to simulate unseen "in-the-wild" generators, including older models, GANs, and alternative architectures: DALL·E 1 (DE1), Deep AI (DPA), Hotpot AI (HPA), NVIDIA Sana PAG (NVS), Stable Cascade (SCD), Stable Diffusion Attend&Excite (SAE), StyleGAN (SG1), StyleGAN2 (SG2), StyleGAN3 (SG3), and Tencent Hunyuan (THY).

Combined with post-processed variants, the dataset contains a grand total of 50,000 images, focusing exclusively on head-and-shoulder portraits of synthetic people.

Examples of Closed Set images generated from the exact same text prompt across 10 different generators.

Key Insights and Ideas

  • The Artifact Fragility Dilemma: CNN-based models achieve near-perfect accuracy (100%) on pristine, unedited images because they latch onto low-level hardware and software artifacts (the "fingerprints" of the generator). However, common editing operations like blurring or compression wipe away these micro-traces, causing CNN performance to plummet.
  • Semantics Over Signatures: Vision Transformers (VTC) prove to be significantly more robust against post-processing. Because ViTs leverage self-attention to capture long-range dependencies and higher-level semantic structures, they maintain high accuracy even when low-level artifacts are destroyed.
  • The Open-Set Challenge: Distinguishing known models from entirely unknown generators (open-set rejection) remains incredibly difficult. While some models can successfully reject unknown models under ideal conditions, their ability to do so degrades rapidly when any form of image editing is introduced.

Key Results

  • Closed-Set Attribution: On plain images, CNNs (EfficientNetB4, XceptionNet) achieved 100% balanced accuracy. However, after 3 steps of post-processing, their accuracy dropped sharply to ~59%. Conversely, the Vision Transformer Classifier (VTC) started at 95.80% on plain images but maintained an impressive 88.67% accuracy after 3 steps of editing.
  • Open-Set Rejection: In a multi-class setup with a rejection threshold (aiming for a 5% false open-set detection rate on validation data), XceptionNet achieved a perfect Correct Classification Rate (CCR) of 1.0 across all known classes and a Correct Rejection Rate (CRR) of 0.70 for unknown open-set models on plain images.
Fig. 4: Confusion matrices obtained by CLIP+MLP (a), DE-FAKE (b), XceptionNet (c), and VTC (d) on the open set attribution with rejection threshold, without post-processing. The generator-classes are represented by the three-letter codes defined in Section II-B.

Limitations and Future Work

The primary limitation of the WILD dataset is its narrow semantic scope: it consists exclusively of head-and-shoulder portraits of synthetic human faces generated from fixed prompt structures. While this controls for content bias, real-world forensic scenarios involve diverse objects, landscapes, and text.

The authors suggest that future work should expand the dataset's diversity and explore how prompt semantics explicitly alter the artifacts left by different generators. Additionally, they highlight the need for explainability studies to better understand exactly what features ViT-based models rely on when performing robust source attribution.

Summary updated

Abstract

Synthetic image source attribution is an open challenge, with an increasing number of image generators being released yearly. The complexity and the sheer number of available generative techniques, as well as the scarcity of high-quality open source datasets of diverse nature for this task, make training and benchmarking synthetic image source attribution models very challenging. WILD is a new in-the-Wild Image Linkage Dataset designed to provide a powerful training and benchmarking tool for synthetic image attribution models. The dataset is built out of a closed set of 10 popular commercial generators, which constitutes the training base of attribution models, and an open set of 10 additional generators, simulating a real-world in-the-wild scenario. Each generator is represented by 1,000 images, for a total of 10,000 images in the closed set and 10,000 images in the open set. Half of the images are post-processed with a wide range of operators. WILD allows benchmarking attribution models in a wide range of tasks, including closed and open set identification and verification, and robust attribution with respect to post-processing and adversarial attacks. Models trained on WILD are expected to benefit from the challenging scenario represented by the dataset itself. Moreover, an assessment of seven baseline methodologies on closed and open set attribution is presented, including robustness tests with respect to post-processing.
BibTeX
@misc{bongini2025wild,
  author = {Pietro Bongini and Sara Mandelli and Andrea Montibeller and Mirko Casu and Orazio Pontorno and Claudio Vittorio Ragaglia and Luca Zanchetta and Mattia Aquilina and Taiba Majid Wani and Luca Guarnera and Benedetta Tondi and Giulia Boato and Paolo Bestagini and Irene Amerini and Francesco De Natale and Sebastiano Battiato and Mauro Barni},
  title = {{WILD: a new in-the-Wild Image Linkage Dataset for synthetic image attribution}},
  year = {2025},
  eprint = {2504.19595},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2504.19595},
}
Figure