Detecting Deepfake Videos using Attribution-Based Confidence Metric

Steven Lawrence Fernandes, Sunny Raj, Rickard Ewetz, Jodh Singh Pannu, Sumit Kumar Jha, Eddy Ortiz, Iustina Vintila, Margaret Salter

Abstract

As Generative Adversarial Networks (GANs) evolve, deepfake videos are becoming increasingly difficult to distinguish from authentic footage. Traditional detection methods often rely on access to training datasets or specific GAN architectures. This paper presents a novel "data-blind" detection approach using the Attribution-Based Confidence (ABC) metric. By analyzing the resilience of facial recognition models to feature mutations, the ABC metric identifies deepfakes with over 96% accuracy. Our results demonstrate that original images exhibit significantly higher stability compared to GAN-generated adversarial faces, providing a robust universal signature for authenticity.

Introduction

The rapid advancement of GANs has made deepfakes nearly indistinguishable to the human eye, posing significant security and ethical challenges. Most current detection frameworks suffer from a "bottleneck" where they require massive datasets or calibration on specific fake types. This research addresses the challenge of data-blind detection, aiming for a method that functions in "black box" scenarios where only the trained model is accessible.

Overview of the ABC Metric workflow.

Methodology

The proposed method utilizes the Attribution-Based Confidence (ABC) metric, inspired by the "Dual Process Theory" of cognition, which distinguishes between intuitive and logical processing.

System 1: The Intuitive Model

A standard ResNet50 architecture pre-trained on the VGGFace2 dataset serves as the primary face recognizer. This represents the "fast, intuitive" response of the system, identifying faces based on learned patterns.

System 2: The ABC Metric

The ABC metric acts as a logical check. It employs Shapley values to identify the specific pixels or features (attributions) most critical to the model's decision. This step ensures that the detection is based on the most relevant facial features rather than global image noise.

The Stress Test

The algorithm performs a "stress test" by mutating high-attribution features—replacing them with a baseline, such as a black patch. The core finding is a disparity in resilience:

  • Original Images: Highly resilient; model confidence remains stable even when key features are tweaked.
  • Deepfakes: Brittle; minor changes to key features cause a significant drop in model confidence.
The training and testing pipeline for deepfake detection using the ABC metric.

Datasets and Experimental Setup

The study used three primary sources for original videos: COHFACE, VidTIMIT, and YouTube celebrity footage. Additionally, the authors generated two new deepfake datasets using a commercial tool (deepfakesweb.com). This inclusion of "real-world" deepfakes created by non-experts ensures the model's applicability outside of controlled laboratory environments.

Key Results and Insights

The ABC metric achieved an average validation accuracy exceeding 96% across all datasets. A critical discovery was the "0.94 Threshold": original videos consistently yielded ABC scores above 0.94, while deepfakes fell below this mark.

Unlike global dimensionality reduction techniques, the ABC metric focuses on locally relevant features for specific faces, enhancing robustness against variations in lighting and head poses. Furthermore, by focusing only on high-attribution features, the method maintains computational efficiency by avoiding the sampling of the entire high-dimensional image space.

Discussion and Future Work

The research highlights that deep neural networks are inherently more stable when processing real human faces than GAN-generated ones. Future work will explore more sophisticated mutation baselines, such as blurred or average faces, and the integration of physiological signals (e.g., heart rate extraction) to further bolster detection certainty. While efficient, the calculation of Shapley values remains a target for optimization to support real-time live-stream detection.

Summary updated

Abstract

Recent advances in generative adversarial networks have made detecting fake videos a challenging task. In this paper, we propose the application of the state-of-the-art attribution based confidence (ABC) metric for detecting deepfake videos. The ABC metric does not require access to the training data or training the calibration model on the validation data. The ABC metric can be used to draw inferences even when only the trained model is available. Here, we utilize the ABC metric to characterize whether a video is original or fake. The deep learning model is trained only on original videos. The ABC metric uses the trained model to generate confidence values. For, original videos, the confidence values are greater than 0.94.
BibTeX
@inproceedings{fernandes2020detecting,
  author = {Steven Lawrence Fernandes and Sunny Raj and Rickard Ewetz and Jodh Singh Pannu and Sumit Kumar Jha and Eddy Ortiz and Iustina Vintila and Margaret Salter},
  title = {{Detecting Deepfake Videos using Attribution-Based Confidence Metric}},
  booktitle = {Workshops @ CVPR},
  year = {2020},
  doi = {10.1109/cvprw50498.2020.00162},
}
Figure