Is It Certainly a Deepfake? Reliability Analysis in Detection & Generation Ecosystem

Neslihan Köse, Anthony Rhodes, Umur Aybars Çiftçi, İlke Demir

Abstract

As generative models advance in quality, deepfakes increasingly contribute to online mistrust. While many detectors have been proposed, their misuse—claiming fake content as real or vice versa—can exacerbate misinformation. This research presents the first comprehensive uncertainty analysis of deepfake detectors, investigating how generative artifacts influence prediction confidence. By leveraging Bayesian Neural Networks and Monte Carlo dropout, the study quantifies aleatoric and epistemic uncertainties across diverse architectures. The findings reveal that uncertainty patterns can serve as a "fingerprint" for source detection and that biological-signal-based detectors offer superior reliability compared to standard image-based models.

The Reliability Gap in Deepfake Detection

The "arms race" between deepfake generators and detectors has traditionally focused on a single metric: accuracy. However, a detector that is 99% accurate in a laboratory setting might be dangerously overconfident when encountering a novel generation technique. Current detectors often function as "black boxes," providing no indication of internal doubt when they misclassify content. This lack of reliability analysis hinders the deployment of detection systems in high-stakes environments like newsrooms or legal proceedings. This study addresses this gap by systematically investigating prediction uncertainty—understanding when and why detectors exhibit doubt.

Quantifying Uncertainty: Methods and Metrics

The researchers evaluated six detector architectures, categorized into two groups:

  1. "Blind" Detectors: Standard computer vision models (e.g., ResNet, Xception, EfficientNet) that search for generic image artifacts.
  2. Biological Detectors: Specialized models (e.g., FakeCatcher) that detect human-specific signals, such as photoplethysmography (PPG) heart rate data or subtle muscle motions.

To measure uncertainty, the study transitioned from deterministic models to Bayesian Neural Networks (BNNs) using Monte Carlo (MC) Dropout. This approach allows models to produce a distribution of possible answers rather than a single point estimate, enabling the calculation of:

  • Aleatoric Uncertainty: Doubt arising from inherent noise in the data (e.g., low-quality or blurry video).
  • Epistemic Uncertainty: Doubt arising from the model's lack of knowledge (e.g., encountering a previously unseen generation method).
Comparison of uncertainty distributions and accuracy vs. retention fraction for ResNet18 (top) and FakeCatcher (bottom) across different generators.

Key Insights: Biological Signals and Forensic Signatures

Biological Signals = Better Calibration

A primary finding is that biological detectors are significantly more "honest." When converted to Bayesian frameworks, their accuracy remains stable, whereas "blind" detectors often experience a massive performance drop. This suggests that identifying "humanness" is a more robust strategy than searching for specific "AI-ness" artifacts.

Uncertainty as a Forensic Signature

Different AI generators leave unique "residues." Because detectors struggle with these residues in predictable ways, the resulting uncertainty manifold contains enough information to perform source detection—identifying which specific AI model created the fake.

Spatial Uncertainty and the "Skull" Pattern

The researchers introduced Uncertainty Maps, a visualization tool that highlights which pixels cause the most model confusion. For certain generators, such as Neural Textures, uncertainty concentrates around the mouth and facial boundaries, often creating a "skull-like" pattern on the cheeks and forehead where artifacts are less prevalent.

Uncertainty maps localizing prediction confidence at the pixel level, revealing distinct patterns correlated with generator-specific artifacts.

Performance and Robustness

  • Stability vs. Complexity: While complex models like EfficientNet achieved high raw accuracy, they showed poor uncertainty calibration, often being "confidently wrong." EfficientNet's accuracy dropped from 99.38% to 90.93% in a Bayesian framework, while the biological FakeCatcher remained stable at ~98%.
  • Adversarial Vulnerability: Even Bayesian models remain susceptible to adversarial attacks. Accuracy dropped by over 93% on average under attack; however, uncertainty levels spiked during these events, providing a potential "alarm" for detecting adversarial manipulation.

Conclusion and Future Work

The primary trade-off for these reliable models is computational cost, as Bayesian inference requires multiple forward passes. Future research may explore distillation methods to optimize these models for real-time use. As the landscape shifts toward diffusion-based deepfakes, uncertainty quantification must become a fundamental requirement for any trustworthy detection system, ensuring that we not only detect fakes but understand the limits of our detection capabilities.

Summary updated

Abstract

As generative models are advancing in quality and quantity for creating synthetic content, deepfakes begin to cause online mistrust. Deepfake detectors are proposed to counter this effect, however, misuse of detectors claiming fake content as real or vice versa further fuels this misinformation problem. We present the first comprehensive uncertainty analysis of deepfake detectors, systematically investigating how generative artifacts influence prediction confidence. As reflected in detectors' responses, deepfake generators also contribute to this uncertainty as their generative residues vary, so we cross the uncertainty analysis of deepfake detectors and generators. Based on our observations, the uncertainty manifold holds enough consistent information to leverage uncertainty for deepfake source detection. Our approach leverages Bayesian Neural Networks and Monte Carlo dropout to quantify both aleatoric and epistemic uncertainties across diverse detector architectures. We evaluate uncertainty on two datasets with nine generators, with four blind and two biological detectors, compare different uncertainty methods, explore region- and pixel-based uncertainty, and conduct ablation studies. We conduct and analyze binary real/fake, multi-class real/fake, source detection, and leave-one-out experiments between the generator/detector combinations to share their generalization capability, model calibration, uncertainty, and robustness against adversarial attacks. We further introduce uncertainty maps that localize prediction confidence at the pixel level, revealing distinct patterns correlated with generator-specific artifacts. Our analysis provides critical insights for deploying reliable deepfake detection systems and establishes uncertainty quantification as a fundamental requirement for trustworthy synthetic media detection.
BibTeX
@misc{kose2025is,
  author = {Neslihan Köse and Anthony Rhodes and Umur Aybars Çiftçi and İlke Demir},
  title = {{Is It Certainly a Deepfake? Reliability Analysis in Detection \& Generation Ecosystem}},
  year = {2025},
  eprint = {2509.17550},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2509.17550},
}
Figure