Is JPEG AI going to change image forensics?

Edoardo Daniele Cannas, Sara Mandelli, Natasa Popovic, Ayman Alkhateeb, Alessandro Gnutti, Paolo Bestagini, Stefano TubaroarXiv 2024 · DOI · arXiv

Abstract

JPEG AI, the first international image compression standard based on neural networks, introduces "counter-forensic" artifacts that challenge existing image authentication tools. This study demonstrates how JPEG AI compression causes state-of-the-art detectors to misclassify pristine images as deepfakes and severely degrades splicing localization performance.

Introduction

For decades, image forensics has relied on the fact that different processes—like taking a photo, saving a JPEG, or "Photoshopping" an area—leave distinct mathematical footprints. Modern deepfake detectors and splicing localizers are trained to find the specific artifacts left by neural networks (like GANs or diffusion models), such as checkerboard patterns caused by upsampling layers.

Enter JPEG AI. Unlike traditional JPEG, which uses Discrete Cosine Transform (DCT), JPEG AI uses end-to-end deep learning (autoencoders) to compress images. The authors identify a looming crisis: the very "upsampling" traces that forensic tools use to catch deepfakes are now being introduced by the compression standard itself. If every "pristine" image starts looking like a deepfake to a computer, our ability to trust digital media collapses.

Methodology

The researchers conducted a comprehensive "stress test" of current forensic technology against the new standard using the official JPEG AI Reference Software (v7.0). They evaluated a battery of state-of-the-art (SOTA) detectors, including CNN-based models, Transformer-based architectures, and CLIP-based detectors. The study utilized a massive array of standard benchmarks for deepfake detection (FFHQ, LSUN, CelebA, etc.) and splicing localization (CASIA, Columbia, etc.).

The "Synthetic" Footprint of JPEG AI

JPEG AI doesn't just blur images; it "overwrites" them with neural artifacts. Fourier analysis reveals that JPEG AI leaves grid-like peaks in the power spectrum—footprints almost identical to those found in GAN-generated deepfakes.

Fourier analysis showing grid-like artifacts in JPEG AI compressed images compared to pristine versions.

As compression becomes more aggressive (lower BPP), these artifacts become more pronounced, leading to a dramatic shift in how forensic models perceive the image. Traditional JPEG compression usually hides evidence of tampering by smoothing it out. JPEG AI does the opposite: it adds new evidence that looks like tampering, leading to massive false positive rates.

Impact on Forensic Performance

The experimental results highlight a significant vulnerability in current forensic tools. At low bitrates (BPP = 0.12), the False Positive Rate (FPR) for several SOTA detectors skyrocketed to over 90%. Essentially, the detectors thought almost every real image was a deepfake.

Deepfake likelihood scores shift toward 'fake' with higher compression, and splicing localization maps become noisy and unreliable.

For image splicing (where one part of an image is moved to another), JPEG AI compression caused localization maps to become "noisy." Detectors flagged pristine pixels as manipulated and missed actual manipulated areas. While retraining detectors with JPEG AI samples improved performance, it didn't fully solve the problem. Balanced Accuracy (BA) remained significantly lower than on uncompressed data, suggesting that JPEG AI artifacts are fundamentally difficult to distinguish from malicious edits.

Conclusion and Future Directions

As JPEG AI rolls out globally, the forensic community faces a race against time. Because JPEG AI is a standardized format, its decoder architecture is fixed, meaning these forensic vulnerabilities will likely be universal across all devices and software that adopt the standard.

Future work must focus on developing robust-by-design detectors invariant to neural compression traces and creating large-scale datasets of JPEG AI-compressed images to harden future forensic tools. Additionally, exploring "proactive" measures like the Content Authenticity Initiative (CAI) or digital watermarking will be crucial to supplement algorithmic detection.

Summary updated

Abstract

In this paper, we investigate the counter-forensic effects of the new JPEG AI standard based on neural image compression, focusing on two critical areas: deepfake image detection and image splicing localization. Neural image compression leverages advanced neural network algorithms to achieve higher compression rates while maintaining image quality. However, it introduces artifacts that closely resemble those generated by image synthesis techniques and image splicing pipelines, complicating the work of researchers when discriminating pristine from manipulated content. We comprehensively analyze JPEG AI's counter-forensic effects through extensive experiments on several state-of-the-art detectors and datasets. Our results demonstrate a reduction in the performance of leading forensic detectors when analyzing content processed through JPEG AI. By exposing the vulnerabilities of the available forensic tools, we aim to raise the urgent need for multimedia forensics researchers to include JPEG AI images in their experimental setups and develop robust forensic techniques to distinguish between neural compression artifacts and actual manipulations.
BibTeX
@misc{cannas2024is,
  author = {Edoardo Daniele Cannas and Sara Mandelli and Natasa Popovic and Ayman Alkhateeb and Alessandro Gnutti and Paolo Bestagini and Stefano Tubaro},
  title = {{Is JPEG AI going to change image forensics?}},
  year = {2024},
  eprint = {2412.03261},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2412.03261},
}
Figure