Fake or JPEG? Revealing Common Biases in Generated Image Detection Datasets

Patrick Grommelt, Louis Weiss, Franz-Josef Pfreundt, Janis Keuper

Introduction

As generative models like GANs and Diffusion Models become more sophisticated, the need for reliable "fake image" detectors has grown. However, a persistent issue in the field is poor cross-generator performance: detectors often perform perfectly on the specific generator they were trained on but fail when tested on images from a different model. This paper hypothesizes that this failure is due to "shortcuts" or biases within detection datasets—specifically JPEG compression and image size—rather than the inherent difficulty of the task.

Methodology

The researchers employed a systematic approach to expose and mitigate these biases using the GenImage benchmark, which contains over a million images from eight different generators.

  1. Replication: The authors first successfully replicated the baseline results of the GenImage paper to establish a fair comparison.
  2. Sensitivity Testing: They evaluated detectors against varying levels of JPEG compression and different image sizes to see if accuracy correlated with these non-semantic features.
  3. Dataset Unbiasing: They created "unbiased" subsets by matching JPEG quality factors and image dimensions between natural and generated classes.
  4. Retraining: Standard architectures (ResNet50 and Swin-T) were retrained on these constrained subsets to force the models to learn structural generation artifacts.
The authors replicated the original GenImage results, showing high accuracy on the diagonal (same-generator testing) but poor generalization to other models.

Key Insights: The Shortcut Problem

The investigation revealed two primary biases that act as proxies for "fakeness":

JPEG Compression Bias

In many datasets, natural images are JPEGs while generated ones are uncompressed PNGs. Detectors quickly learn that "uncompressed = fake." When AI-generated images are compressed, the detectors' accuracy plummets because the shortcut is removed. This suggests that the models are often just "JPEG detectors" in disguise.

Detector performance drops to near-random levels when tested on compressed images, exposing a heavy reliance on compression artifacts.

Image Size Bias

Generators often produce images of a fixed size (e.g., 512x512), whereas natural images in datasets like ImageNet vary. Detectors were found to be significantly less accurate on natural images that happened to match the fixed dimensions of the AI images in the training set, indicating that size is used as a proxy for classification.

Results and Impact

By removing these easy shortcuts, models are forced to identify actual structural and spectral artifacts inherent to the generation process. This leads to significantly more robust models:

  • Generalization Boost: Removing size and compression biases led to an average increase of over 11 percentage points in cross-generator accuracy for both ResNet50 and Swin-T.
  • State-of-the-Art Performance: A Swin-T detector trained on unbiased data achieved an average cross-generator accuracy of 85.83%, outperforming specialized methods like GenDet.
  • Robustness: Unlike baseline models, those trained on unbiased data remained accurate even when images were subjected to further post-processing.

Conclusion

This work demonstrates that common dataset biases significantly inflate the perceived performance of fake image detectors while hindering their real-world utility. By cleaning the GenImage benchmark, the authors have provided a path toward more generalizable and reliable detection. Future work should continue to align natural and synthetic images in terms of content and source to eliminate remaining semantic biases.

Summary updated

BibTeX
@inproceedings{grommelt2024fake,
  author = {Patrick Grommelt and Louis Weiss and Franz-Josef Pfreundt and Janis Keuper},
  title = {{Fake or JPEG? Revealing Common Biases in Generated Image Detection Datasets}},
  booktitle = {Workshops @ ECCV},
  year = {2024},
  doi = {10.1007/978-3-031-92089-9\_6},
}

Loading related content...

Figure