Towards reliable deepfake detection from uncertainty calibration perspective

Xue-sheng Jin, Weinan Guan, Wei Wang, Jing DongVisual 2025 · DOI

Abstract

The rapid development of face forgery technology has raised significant concerns regarding information security. While deepfake detection has proven to be an effective countermeasure, existing methods typically focus on binary classification accuracy, leaving predictive reliability largely unknown. Models often exhibit "overconfidence," assigning extremely high probabilities to incorrect predictions, especially when encountering unseen content. This paper introduces the concept of deepfake detection calibration using Packed-Ensembles (PE). By dividing a detector into multiple subnetworks, PE allows the model to focus on diverse forgery traces, providing reliable uncertainty estimation without incurring additional computational costs. Comprehensive experiments demonstrate that this method effectively mitigates overconfidence and enhances detection reliability while maintaining strong generalization performance.

The Challenge: Overconfidence in Deepfake Detection

Most research in deepfake detection focuses on binary accuracy—classifying an image as "real" or "fake." However, in high-stakes environments like legal proceedings or financial verification, knowing when a model is likely to be wrong is as crucial as the prediction itself.

Current detectors suffer from a phenomenon known as overconfidence. When a model encounters a forgery type it wasn't trained on, it often outputs an extreme probability (e.g., 0.99) even when its prediction is incorrect. This indicates a failure in model calibration—the alignment between a model’s predicted probability and the actual likelihood of correctness.

Distribution of confidence scores showing a high frequency of samples with near-maximum confidence, indicative of the overconfidence problem.

Methodology: Packed-Ensembles for Deepfake Detection (PE-DD)

To address these issues, the authors propose the PE-DD framework, which rests on two innovative pillars:

1. Packed-Ensembles (PE)

Traditional ensembles improve reliability by training multiple separate models, but they are computationally expensive. PE uses sparse grouped convolutions to "pack" multiple subnetworks into a single architecture. By dividing the network along channel dimensions, each subnetwork is forced to learn different forgery traces. This provides the diversity of an ensemble with the computational footprint of a single model.

2. High-Frequency Feature Injection

Deepfakes often leave subtle traces in the high-frequency domain that are invisible to the naked eye. The authors utilize SRM (Steganalysis-Rich Model) filters to extract edge and noise patterns. These features are concatenated with original image channels, helping the subnetworks detect imperceptible texture anomalies and artifacts.

Experimental Evaluation

The method was evaluated on the FaceForensics++ (FF++) dataset for training, with extensive testing on out-of-distribution (OOD) benchmarks including Celeb-DF v2, DFDC, Wild Deepfake (WDF), and AI-generated images (GenImage).

Key Results and Reliability

The primary metric for reliability is the Expected Calibration Error (ECE). A perfectly calibrated model would have an ECE of zero, where its confidence matches its accuracy exactly.

A reliability diagram showing the gap between predicted confidence and actual accuracy. The goal of calibration is to minimize this 'Gap'.
  • Calibration Gains: On the FF++ dataset, PE-DD reduced the ECE from 0.085 (standard model) to 0.020, representing a massive improvement in reliability.
  • Robustness to Noise: Under real-world "distribution shifts" like Gaussian noise or JPEG compression, PE-DD maintained significantly higher AUC and lower ECE than standard models, which typically become wildly overconfident when faced with noise.
  • Efficiency: PE-DD achieved better calibration than traditional Deep Ensembles (DE) while using only 25% of the parameters and maintaining higher throughput (images per second).

Conclusion

PE-DD demonstrates that deepfake detectors do not have to sacrifice accuracy for reliability. By leveraging internal subnetworks and high-frequency features, the framework provides a computationally efficient way to estimate uncertainty and mitigate overconfidence. This shift toward calibrated detection is a vital step toward deploying trustworthy AI in sensitive security domains.

Summary updated

Abstract

Abstract The rapid development of face forgery technology has raised concerns among researchers about information security. To mitigate the malicious abuse of the technology, deepfake detection has proven to be an effective countermeasure. However, existing methods only provide a true-false and binary classification, and they lack research on uncertainty estimation, which leads to the predictive reliability being unknown. The model uncertainty reflects its understanding of knowledge. Even when faced with unseen deepfake content, a detector without awareness of its cognitive limitations may still output incorrect predictions with high confidence. The detector assigns extremely high probabilities to samples, which is referred to a phenomenon known as overconfidence. In this paper, we introduce the novel concept of deepfake detection calibration, which focuses on aligning predicted probability with actual distribution to alleviate the overconfidence and estimate the uncertainty. Specifically, we introduce an innovative calibration method called packed-ensembles (PE), which divides a detector into multiple subnetworks. PE allows each subnetwork to focus on different forgery traces, resulting in reliable predictions without incurring additional computational costs. Comprehensive experiments demonstrate that our method effectively mitigates the overconfidence in deepfake detection while maintaining generalization, and significantly enhances detection reliability.
BibTeX
@inproceedings{jin2025towards,
  author = {Xue-sheng Jin and Weinan Guan and Wei Wang and Jing Dong},
  title = {{Towards reliable deepfake detection from uncertainty calibration perspective}},
  booktitle = {Visual},
  year = {2025},
  doi = {10.1007/s44267-025-00100-2},
}
Figure