MIRAGE: Towards AI-Generated Image Detection in the Wild

Cheng Xia, Manxi Lin, Jiexiang Tan, Xiaoxiong Du, Yang Qiu, Junjun Zheng, Xiangheng Kong, Yuning Jiang, Bo ZhengAAAI 2025 · OpenReview

TL;DR

This paper introduces MIRAGE, a new benchmark for detecting AI-generated images in real-world, noisy conditions, and MIRAGE-R1, a novel vision-language model that uses adaptive, reflective reasoning to achieve state-of-the-art performance. This work is crucial for combating the growing threat of sophisticated AI forgeries that evade current detection methods.

What Problem Does This Paper Address?

The proliferation of AI-generated images (AIGI) poses a significant threat to information security and public trust. While existing AIGI detectors perform well in controlled lab settings, they struggle to generalize to "in-the-wild" scenarios. These real-world images are complex: they can range from obviously fake to highly realistic, often modified through multiple generative models and post-processing steps. This makes them difficult to detect and poses challenges for current models that often rely on domain-specific artifacts or struggle with explainability and efficiency. The authors aim to bridge this gap by developing a more robust detection method and a benchmark that accurately reflects these real-world complexities.

Methods

The core of the proposed solution is MIRAGE-R1, a vision-language model (VLM) that employs a novel "Heuristic-to-Analytic Reasoning" (HA-R) framework. This framework mimics human cognitive processes by first generating a quick "impression" (heuristic answer) and then, if confidence is low, engaging in deeper, "reflective" reasoning to arrive at a more robust final answer. This adaptive approach balances speed and accuracy. MIRAGE-R1 is trained in two stages:

  1. Supervised Fine-Tuning (SFT): A "cold-start" phase to teach the model the HA-R format.
  2. Reinforcement Learning with Verifiable Reward (RLVR): This stage uses a custom reward function that includes a "confidence-aware" reward to encourage calibrated initial judgments and a "soft" reward for reflective reasoning, promoting more nuanced and accurate analysis.

The model also incorporates an "inference-time adaptive thinking" strategy, where it dynamically decides whether to perform deep reasoning based on the confidence of its initial assessment, thus optimizing computational resources.

Datasets

The paper introduces MIRAGE, a new benchmark designed to emulate in-the-wild AIGI detection. It comprises:

  • Human Curation: A large corpus of Internet-sourced AIGI verified by human experts, including "obviously fake" images.
  • Synthesized Dataset: Images created through collaboration between multiple expert generators, simulating realistic in-the-wild AIGI. This includes images from:
  • Vanilla Generators: Standard text-to-image models.
  • Composite Pipelines: Images generated using complex, multi-stage workflows involving various models and post-processing steps (e.g., face restoration, editing).

MIRAGE is partitioned into In-Distribution (ID) and Out-of-Distribution (OOD) splits for comprehensive evaluation. The authors also used the public Chameleon benchmark for cross-validation.

Key Insights and Ideas

  • The "In-the-Wild" Challenge: The paper highlights that real-world AIGI are far more complex than those found in controlled datasets, requiring models to handle noise, diverse quality, and sophisticated generation pipelines.
  • Adaptive Reasoning is Key: MIRAGE-R1's adaptive approach, which balances quick heuristic judgments with deeper reflective analysis, is crucial for achieving both efficiency and accuracy in real-world scenarios. This avoids the "over-thinking" problem of purely analytic models and the unreliability of purely heuristic ones.
  • Reflective Reasoning Matters: The inclusion of a reflective step, where the model critically re-evaluates its initial assessment, significantly improves generalization and robustness, especially for challenging OOD cases.
  • Dataset Design is Crucial: The MIRAGE benchmark, with its diverse sources and complex generation pipelines, is shown to better prepare models for in-the-wild AIGI detection compared to existing benchmarks.

Key Results

  • MIRAGE-R1 achieves state-of-the-art performance on the MIRAGE benchmark, outperforming existing detectors by 5% on the ID set and 10% on the OOD set.
  • It also sets a new state-of-the-art on the challenging Chameleon benchmark.
  • The model demonstrates strong generalization capabilities across various generation patterns and is robust to common image corruptions like JPEG compression and Gaussian blur, even without specific training on these augmentations.
  • Qualitative results show MIRAGE-R1's ability to correctly identify fake images and provide detailed, accurate explanations, often correcting initial incorrect heuristic judgments through its reflective process.

Limitations and Future Work

  • Potential Biases: The authors acknowledge that future work could focus on mitigating potential biases inherited from the backbone VLMs and VLMs used for annotation.
  • Scaling and Refinement: Future research could explore scaling data annotation efforts and further refining the adaptive reasoning mechanism.
  • Mitigating Over-Thinking: While MIRAGE-R1's adaptive strategy helps, the paper notes that "over-thinking" can still occur in some cases, suggesting ongoing research into fine-tuning the confidence thresholds and reasoning triggers.

Summary updated

Abstract

The spreading of AI-generated images (AIGI), driven by advances in generative AI, poses a significant threat to information security and public trust. Existing AIGI detectors, while effective against images in clean laboratory settings, fail to generalize to in-the-wild scenarios. These real-world images are noisy, varying from ``obviously fake" images to realistic ones derived from multiple generative models and further edited for quality control. We address in-the-wild AIGI detection in this paper. We introduce Mirage, a challenging benchmark designed to emulate the complexity of in-the-wild AIGI. Mirage is constructed from two sources: (1) a large corpus of Internet-sourced AIGI verified by human experts, and (2) a synthesized dataset created through the collaboration between multiple expert generators, closely simulating the realistic AIGI in the wild. Building on this benchmark, we propose Mirage-R1, a vision-language model with heuristic-to-analytic reasoning, a reflective reasoning mechanism for AIGI detection. Mirage-R1 is trained in two stages: a supervised-fine-tuning cold start, followed by a reinforcement learning stage. By further adopting an inference-time adaptive thinking strategy, Mirage-R1 is able to provide either a quick judgment or a more robust and accurate conclusion, effectively balancing inference speed and performance. Extensive experiments show that our model leads state-of-the-art detectors by 5% and 10% on Mirage and the public benchmark, respectively. The benchmark and code will be made publicly available.
BibTeX
@misc{xia2025mirage,
  author = {Cheng Xia and Manxi Lin and Jiexiang Tan and Xiaoxiong Du and Yang Qiu and Junjun Zheng and Xiangheng Kong and Yuning Jiang and Bo Zheng},
  title = {{MIRAGE: Towards AI-Generated Image Detection in the Wild}},
  year = {2025},
  eprint = {2508.13223},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2508.13223},
}
Figure