CSTAN: A Deepfake Detection Network with CST Attention for Superior Generalization

Rui Yang, Kang Soo You, Cheng Pang, Xiaonan Luo, Rushi Lan

TL;DR

This paper introduces CSTAN, a novel deepfake detection network designed for superior generalization across different datasets. By combining a dynamic feature extraction backbone (OD-ResNet-34) with a Channel-Spatial-Triplet (CST) attention mechanism, CSTAN effectively captures subtle forgery artifacts, leading to significantly improved performance on unseen data compared to existing methods.

What Problem Does This Paper Address?

The proliferation of highly realistic deepfake technology poses a significant threat to security systems, particularly facial recognition. While current deep learning-based deepfake detection models achieve high accuracy on the datasets they are trained on (intra-dataset performance), they often struggle to generalize to new, unseen datasets or different types of deepfake generation techniques. This lack of generalization is a critical bottleneck for real-world deployment, as it means models trained on one set of fake videos might fail when encountering slightly different ones. The paper aims to bridge this gap by developing a model that can better identify the fundamental differences between real and fake facial features, thereby enhancing its robustness and adaptability.

Methods

The proposed CSTAN model is built upon two main innovations:

  • OD-ResNet-34 Backbone: This is a modified ResNet-34 architecture that incorporates Omni-dimensional Dynamic Convolution (ODConv). ODConv dynamically adjusts convolutional kernels based on input data, allowing the network to adapt to varying feature characteristics and improving its ability to generalize across different datasets. This contrasts with standard convolutional layers, which have fixed weights.
  • Channel-Spatial-Triplet (CST) Attention Mechanism: This novel attention module is designed to enhance the detection of subtle forgery artifacts. It combines channel attention (similar to SENet) and triplet attention, focusing on capturing local information by analyzing feature channels and spatial correlations across three different scales. This multi-scale approach helps the model pinpoint and emphasize regions likely to contain deepfake manipulations.

The model also employs a multi-scale detection module with anchor points and a loss function that combines classification and detection losses to guide the learning process.

Datasets

  • Training: FF++ dataset.
  • Testing/Evaluation: Celeb-DF-v1 and Celeb-DF-v2 datasets.

The paper notes that the FF++ dataset was used for initial training and ablation studies, while Celeb-DF-v1 and Celeb-DF-v2 were used to evaluate the model's cross-dataset generalization capabilities.

Key Insights and Ideas

  • The Importance of Dynamic Adaptability: The success of ODConv in improving the backbone network highlights that static feature extractors are a limitation for deepfake detection. Dynamically adapting convolutional kernels to input data is crucial for better generalization.
  • Multi-Scale Attention for Subtle Artifacts: Deepfake artifacts can be subtle and manifest at various scales. The CST attention mechanism's ability to capture these artifacts across different scales is key to its improved performance.
  • Generalization is the Real Challenge: While achieving high accuracy on a single dataset is achievable, the true test of a deepfake detection system lies in its ability to perform well on diverse and unseen data. This paper directly tackles this critical challenge.

Key Results

The CSTAN model demonstrated superior performance in cross-dataset detection tasks.

  • On the FF++ dataset: CSTAN achieved an accuracy of 99.96%, outperforming the baseline by 0.72%.
  • On the Celeb-DF-v1 dataset: CSTAN achieved an accuracy of 94.24%, a 3.97% improvement over the baseline.
  • On the Celeb-DF-v2 dataset: CSTAN achieved the highest accuracy of 91.75%, a 4.80% improvement over the baseline, and significantly outperformed other state-of-the-art models.
  • Comparative Analysis: When compared to several other deepfake detection models (Multi-task, Xception, DSP-FWA, M2TR, PCL, CADDM, Deepfake Catcher), CSTAN consistently showed superior AUC scores, particularly on the challenging Celeb-DF-v2 dataset.

Limitations and Future Work

  • Limitations: The paper acknowledges that while CSTAN shows improved generalization, further optimization of data augmentation strategies could enhance adaptability to even more diverse datasets.
  • Future Work: The authors suggest exploring more diverse and complex training data to further improve model robustness. Additionally, they propose investigating more effective transfer learning techniques to enhance the model's generalization capabilities across different datasets.

Summary updated

Abstract

With the advancement of deepfake forgery technology, highly realistic fake faces have posed serious security risks to sensor-based facial recognition systems. Recent deepfake detection models mainly use binary classification models based on deep learning. Despite achieving high detection accuracy on intra-datasets, these models lack generalization ability when applied to cross-datasets. We propose a deepfake detection model named Channel-Spatial-Triplet Attention Network (CSTAN), which focuses on the difference between real and fake features, thereby enhancing the generality of the detection model. To enhance the feature-learning ability of the model for image forgery regions, we have designed the Channel-Spatial-Triplet (CST) attention mechanism, which extracts subtle local information by capturing feature channels and the spatial correlation of three different scales. Additionally, we propose a novel feature extraction method, OD-ResNet-34, by embedding ODConv into the feature extraction network to enhance its dynamic adaptability to data features. Trained on the FF++ dataset and tested on the Celeb-DF-v1 and Celeb-DF-v2 datasets, the experimental results show that our model has stronger generalization ability in cross-datasets than similar models.
BibTeX
@inproceedings{yang2024cstan,
  author = {Rui Yang and Kang Soo You and Cheng Pang and Xiaonan Luo and Rushi Lan},
  title = {{CSTAN: A Deepfake Detection Network with CST Attention for Superior Generalization}},
  booktitle = {Sensors},
  year = {2024},
  doi = {10.3390/s24227101},
}
Figure