NeXT-IMDL: Build Benchmark for NeXT-Generation Image Manipulation Detection & Localization

Yifei Li, Haoyuan He, Yu Zheng, Bingyao Yu, Wenzhao Zheng, Lei Chen, Jie Zhou, Jiwen Lu

Abstract

NeXT-IMDL is a massive new benchmark designed to expose the "benchmark illusion" in image manipulation detection, where models appear robust on paper but fail in real-world scenarios. By testing across 32 different editing tools and four critical axes—models, types, semantics, and granularity—the benchmark reveals that current state-of-the-art detectors are surprisingly fragile when faced with unseen AI-generated content.

Introduction

The rise of user-friendly generative AI (AIGC) has made high-quality image manipulation accessible to everyone, creating a major challenge for digital forensics. While many Image Manipulation Detection and Localization (IMDL) models report high scores, they often suffer from a "benchmark illusion." They perform well on the specific datasets they were trained on but crumble when encountering new editing techniques or different types of content. Existing benchmarks are often too narrow, focusing on a few manipulation types or models. This paper addresses the need for a truly generalizable and diagnostic evaluation framework to guide the development of more robust, next-generation forensics tools.

Methods and Framework

The authors propose a comprehensive framework that decouples AIGC-based manipulations into four fundamental axes:

  1. Editing Models: Tests if detectors overfit to the "fingerprints" of specific architectures (e.g., Stable Diffusion vs. FLUX).
  2. Manipulation Types: Evaluates conceptual understanding by testing across removal, replacement, and addition tasks.
  3. Content Semantics: Probes for "shortcut learning" where models might just flag specific objects (like humans or animals) rather than actual artifacts.
  4. Forgery Granularity: Assesses the ability to detect edits of varying sizes, from tiny localized patches to large-scale changes.

To build the NeXT-IMDL dataset, a sophisticated pipeline was used involving diverse proposal generation via four Multimodal Large Language Models (MLLMs) and precise localization using LangSAM.

Overview of the NeXT-IMDL framework, showing the diversity of instructions, models, and the decoupling of factors for comprehensive evaluation.

The NeXT-IMDL Dataset

The primary contribution is the NeXT-IMDL benchmark, which includes 558,269 samples generated using 32 distinct academic and commercial editing tools. These include state-of-the-art open-source models like Stable Diffusion variants and FLUX, alongside commercial tools such as Photoshop and Meitu. Source images were curated from established datasets like MS COCO, Flickr30k, and OpenImages V7. NeXT-IMDL offers superior diversity in terms of model types, conditions (mask, text, reference image), and manipulation types compared to previous benchmarks.

Key Insights and Results

The authors benchmarked 11 representative models using five rigorous cross-dimension protocols, leading to several critical findings:

  • Performance Collapse: While models achieved satisfactory results in their original "in-domain" settings, they experienced dramatic performance degradation in cross-setting evaluations.
  • The "Universal Donor" Effect: Models trained exclusively on object removal tasks show surprisingly good zero-shot performance on replacement and addition tasks, as removal forces the learning of intrinsic "filling" artifacts.
  • Foundation Model Advantage: MaskCLIP, which synergizes CLIP and MAE, consistently outperformed specialized artifact extractors, suggesting a shift toward leveraging large-scale pre-trained models for forensics.
  • Semantic and Granularity Brittleness: Most detectors are heavily biased by the semantics and size of the edits seen during training, leading to significant performance drops when these factors change.
Visualization of model performance (IoU) across different protocols, highlighting the significant decline in cross-scene settings and semantic categories.

Conclusion and Future Work

The study acknowledges that even the best current models are far from being "universal" detectors. Future research should focus on exploring other foundation model combinations (e.g., integrating SAM or Diffusion U-Nets), mitigating semantic bias through adversarial training strategies, and adapting forensic tools to keep pace with the rapid evolution of generative models.

Summary updated

Abstract

The accessibility surge and abuse risks of user-friendly image editing models have created an urgent need for generalizable, up-to-date methods for Image Manipulation Detection and Localization (IMDL). Current IMDL research typically uses cross-dataset evaluation, where models trained on one benchmark are tested on others. However, this simplified evaluation approach conceals the fragility of existing methods when handling diverse AI-generated content, leading to misleading impressions of progress. This paper challenges this illusion by proposing NeXT-IMDL, a large-scale diagnostic benchmark designed not just to collect data, but to probe the generalization boundaries of current detectors systematically. Specifically, NeXT-IMDL categorizes AIGC-based manipulations along four fundamental axes: editing models, manipulation types, content semantics, and forgery granularity. Built upon this, NeXT-IMDL implements five rigorous cross-dimension evaluation protocols. Our extensive experiments on 11 representative models reveal a critical insight: while these models perform well in their original settings, they exhibit systemic failures and significant performance degradation when evaluated under our designed protocols that simulate real-world various generalization scenarios. By providing this diagnostic toolkit and the new findings, we aim to advance the development towards building truly robust, next-generation IMDL models.
BibTeX
@misc{li2025next,
  author = {Yifei Li and Haoyuan He and Yu Zheng and Bingyao Yu and Wenzhao Zheng and Lei Chen and Jie Zhou and Jiwen Lu},
  title = {{NeXT-IMDL: Build Benchmark for NeXT-Generation Image Manipulation Detection \& Localization}},
  year = {2025},
  eprint = {2512.23374},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2512.23374},
}
Figure