EviScene: Beyond "Is It AI?" for Agentic Image Factuality Assessment

ZiAn Wang, Ruiqi Liu, Zhengbo Zhang, Yi Han, Jialiang Shen, Ruoxin Chen, Manni Cui, Ximo Zhu, Jin Jiang, Mingzhe Liu, Linhao Wang, Wenhao Wang, Shu WuarXiv 2026 · arXiv

TL;DR

Researchers have moved beyond simply asking "Is this image AI-generated?" to a more difficult question: "Is the situation depicted in this image factually true?" They introduce EviScene, an autonomous agent that investigates images by interleaving visual inspection with web searches, and EviLens, a massive new benchmark of over 10,000 images designed to test if AI can spot factual errors without being told what to look for.

What Problem Does This Paper Address?

Most visual fact-checking systems are "claim-guided"—you give the AI a photo and a caption like "The Eiffel Tower is in London," and it tells you if that's wrong. But in the real world, misinformation often arrives as a standalone image. We need systems that can perform Image-centered Verification: looking at a picture, identifying the key entities and events, and proactively searching for evidence to confirm or refute the scene.

The challenge is twofold: first, a single glance is rarely enough to verify complex facts; second, the system must align retrieved evidence (like a news report) with specific visual details (like the shape of a trophy or a specific landmark) to ensure they actually refer to the same event.

Figure 1: Claim-guided versus image-centered verification. EviScene alternates evidence retrieval and visual reinspection to refine questions about the depicted event and its context.

Methods

The authors developed EviScene, a multimodal agent based on the ReAct (Reason + Act) framework. Instead of a single-pass judgment, EviScene operates in a loop:

  1. Visual Inspection: It "looks" at the image to identify people, places, and text.
  2. External Retrieval: It uses tools like Google Search and reverse image search to find context.
  3. Refinement: Based on what it finds, it asks new questions (e.g., "Wait, if the 2019 summit was in New York, why is the Eiffel Tower in the background?") and reinspects the image.

To train this agent, they used a two-stage process:

  • Supervised Fine-Tuning (SFT): Learning from "teacher" trajectories generated by a massive 397B-parameter model.
  • Privileged On-Policy Self-Distillation (PSD): This is the "secret sauce." During training, a separate model (Gemini 3.7 Flash) acts as a "hint constructor," identifying where the student agent made a mistake and providing a subtle nudge. The student learns to reach the correct conclusion on its own, without needing those hints at test time.

Datasets

The researchers introduced EviLens, a benchmark of 10,174 images (3,114 supported, 7,060 refuted). It differs from previous datasets by focusing on "world facts" rather than just image quality or AI artifacts.

The dataset covers 12 factual error types grouped into three families:

  • Event & Relation: Invented actions or impossible states.
  • Attribute & Detail: Numerical alterations or identity substitutions.
  • Context: Moving a real event to the wrong time or place.
Figure 2: Factual errors in EviLens: (a) constructed and collected examples with corresponding evidence; (b) the twelve error types grouped into three families.

Key Insights and Ideas

  • Factuality $\neq$ Authenticity: An AI-generated image can depict a true event, and a "real" photograph can be used to lie about a situation (e.g., using an old photo of a protest to represent a current one). EviScene focuses on the truth of the content, regardless of how the pixels were made.
  • The "Aha!" Moment of Re-inspection: The paper highlights that evidence retrieval often changes how we see an image. If a search result says a specific trophy is silver, the agent knows it must go back and check the color of the trophy in the original image.
  • Agentic Intelligence: By giving the model tools (Search, OCR, Image Comparison) and a reasoning loop, it can solve problems that are too complex for a standard "black box" multimodal model.

Key Results

The trained Qwen3.5-9B agent (a relatively compact model) performed remarkably well on the 1,527-image test set:

  • Balanced Accuracy (BAcc): Achieved 79.05%, significantly outperforming base models and even some much larger proprietary models.
  • Strict Evidence Sufficiency Rate (SESR): This is a tough metric that requires the model to get the right answer and cite the correct evidence. EviScene hit 52.26%, a nearly 10-point improvement over the base agent.
  • Tool Importance: Ablation studies showed that removing web search or image retrieval caused performance to plummet, proving that external evidence is the backbone of fact-checking.

Limitations and Future Work

While powerful, EviScene has its limits:

  • Action Budget: The agent is currently limited to 24 actions per investigation to keep costs down.
  • API Dependency: It relies on external services like Serper (for search) and Baidu (for OCR), which can be bottlenecks.
  • Static Images: The current framework is built for still images; the authors suggest that extending this "agentic" approach to video fact-checking is a vital next step.
  • Reasoning Depth: While it catches many errors, extremely subtle "deepfakes" or complex geopolitical nuances might still require human-level reasoning.

Summary updated

Abstract

Determining whether an image is AI-generated does not establish whether the situation it depicts is factually supported. We formulate Image Factuality Assessment: given an image and a generic verification instruction, a system returns a supported or refuted verdict with an evidence-grounded explanation. Without a supplied textual claim, the system must progressively refine factual questions during investigation and align external evidence with the specific situation depicted. We introduce EviLens, an expert-reviewed benchmark of 10,174 supported and refuted images from controlled construction and web collection, covering twelve factual error types and evaluating both verdict accuracy and evidence sufficiency. We develop EviScene, a multimodal ReAct agent that interleaves visual inspection, external retrieval, and evidence analysis, using new findings to refine its questions and revisit relevant image details. We train a compact policy with supervised fine-tuning on reviewed teacher trajectories, followed by privileged on-policy self-distillation applied to verified decisions from the policy's own investigations. On the 1,527-image test set, the trained Qwen3.5-9B agent achieves 79.05% balanced accuracy and a Strict Evidence Sufficiency Rate (SESR) of 52.26%, improving over the same-backbone base agent by 6.71 and 9.37 percentage points, respectively. Its SESR is the highest among the evaluated agents.Code, models, and data will be released at https://github.com/wangzian828/EVISCENE and https://huggingface.co/ZiAnwang/EviScene.
BibTeX
@misc{wang2026eviscene,
  author = {ZiAn Wang and Ruiqi Liu and Zhengbo Zhang and Yi Han and Jialiang Shen and Ruoxin Chen and Manni Cui and Ximo Zhu and Jin Jiang and Mingzhe Liu and Linhao Wang and Wenhao Wang and Shu Wu},
  title = {{EviScene: Beyond "Is It AI?" for Agentic Image Factuality Assessment}},
  year = {2026},
  eprint = {2610.14609},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.14609},
}
Figure