Don't Guess, Escalate: Towards Explainable Uncertainty-Calibrated AI Forensic Agents

Giulia Boato, Andrea Montibeller, Edward J. Delp, Luisa Verdoliva, Daniele MiorandiarXiv 2025 · DOI · arXiv

Abstract

As generative AI models become increasingly sophisticated, traditional forensic methods based on handcrafted cues are failing. This paper proposes a shift toward autonomous AI Forensic Agents that integrate multiple specialized tools, quantify their own uncertainty, and provide human-readable explanations. By adopting a "don't guess, escalate" philosophy, these agents aim to provide the rigour and transparency required for high-stakes legal and journalistic applications.

Introduction: The Forensic Crisis

The explosion of generative AI, including GANs and Diffusion Models, has shattered the assumptions of traditional multimedia forensics. Where investigators once relied on sensor noise or compression artifacts, they now face highly realistic synthetic media that often lacks these low-level traces. Current forensic tools are typically fragmented "one-trick ponies" that offer binary "real/fake" outputs without explaining why a decision was made. This opacity and overconfidence are dangerous in law enforcement or news verification contexts.

The Need for Agentic Forensics

To address these hurdles, we propose a conceptual framework for AI Forensic Orchestrators. Rather than a single algorithm, this is a system-level architecture designed to handle the complexity of modern synthetic media.

Figure 4: Examples of common generative AI artifacts that challenge current detectors, including anatomical distortions (ears, hands), semantic inconsistencies in reflections, and background anomalies.

The proposed framework consists of several key components:

  • Dynamic Orchestration: An AI-driven "brain" that analyzes the media's metadata and semantics to decide which specialized tools (e.g., biometric checks, shadow analysis) to invoke.
  • Uncertainty Calibration: Using statistical methods like conformal prediction to ensure confidence scores reflect real-world probability. If evidence is thin, the agent is designed to abstain—the "escalate" part of our philosophy—rather than guess.
  • Explainable AI (XAI) via MLLMs: Multimodal Large Language Models (MLLMs) are used to translate technical traces into "chain-of-thought" narratives understandable by human investigators.

From Heatmaps to Semantic Narratives

A core insight of this work is that "context is king." Pixel-only approaches are no longer sufficient; forensic agents must reason across modalities and include external context. Current XAI methods like Grad-CAM heatmaps are often uninterpretable. We advocate for "evidence-centric" explanations grounded in quantifiable artifacts.

Figure 3: A semantic artifact where the subject's reflection in the mirror fails to match her physical orientation, demonstrating a high-level inconsistency that requires context-aware reasoning.

By providing a clear "digital chain of custody"—a record of which models were used and why—these agents ensure accountability and auditability.

Challenges and Future Directions

While the vision of AI Forensic Agents is compelling, several challenges remain. Privacy must be balanced against the need for detailed provenance. Furthermore, as forensic agents evolve, AI generators will likely adapt to hide their traces. Finally, MLLMs themselves must be better grounded in physical and signal-processing realities to avoid "hallucinating" forensic justifications. Moving forward, the industry requires unified evaluation metrics, such as the Brier score, to measure the success of probability calibration in forensic systems.

Summary updated

Abstract

AI is reshaping the landscape of multimedia forensics. We propose AI forensic agents: reliable orchestrators that select and combine forensic detectors, identify provenance and context, and provide uncertainty-aware assessments. We highlight pitfalls in current solutions and introduce a unified framework to improve the authenticity verification process.
BibTeX
@misc{boato2025don,
  author = {Giulia Boato and Andrea Montibeller and Edward J. Delp and Luisa Verdoliva and Daniele Miorandi},
  title = {{Don't Guess, Escalate: Towards Explainable Uncertainty-Calibrated AI Forensic Agents}},
  year = {2025},
  eprint = {2512.16614},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2512.16614},
}
Figure