DeepFidelity: Perceptual Forgery Fidelity Assessment for Deepfake Detection

Chunlei Peng, Huiqing Guo, Decheng Liu, Nannan Wang, Ruimin Hu, Xinbo Gao

TL;DR

DeepFidelity shifts deepfake detection from simple binary classification to a continuous "perceptual forgery fidelity" assessment. By accounting for image quality and leveraging facial symmetry through a novel SSAAFormer architecture, the framework achieves state-of-the-art accuracy, particularly on challenging, diverse real-world datasets.

What Problem Does This Paper Address?

Most deepfake detectors treat the problem as a "hard" binary classification: an image is either 100% real or 100% fake. However, this ignores the reality of image quality. Factors like blur, poor lighting, or occlusions can hide the subtle artifacts (forgery clues) that detectors rely on. A low-quality real face might look "suspicious" to a model, while a low-quality fake might lose the very traces that would give it away.

The authors argue that treating all samples as equal leads to sub-optimal training. They address this by introducing perceptual forgery fidelity—a score that reflects both the authenticity and the quality of the face, providing a more nuanced training signal.

Methods

The DeepFidelity framework consists of two main innovations:

  1. Perceptual Forgery Fidelity Mapping: Instead of binary 0/1 labels, the authors map images to a continuous score range [0, 1]. They use a pre-existing face quality assessment tool (Ser-fiq) to score images. Real images are normalized to a range of 0.6 to 1.0, while fake images are mapped to 0.0 to 0.4. This creates a clear "distinguishable gap" while allowing the model to learn that high-quality fakes (closer to 0.4) are different from low-quality ones (closer to 0.0).
  2. SSAAFormer (Symmetric Spatial Attention Augmentation based vision Transformer):
    • Hybrid Architecture: It uses a 4-stage structure. The first two stages use convolutions to capture local features efficiently, while the last two use self-attention for global context.
  3. SSAA Module: In the shallow layers, the authors introduce a module that leverages facial symmetry. It flips the feature maps horizontally and merges them with the original. This allows the network to model "geographic long-distance relationships"—essentially comparing symmetric parts of the face (like the left and right eyes) to find inconsistencies early in the processing pipeline.
Figure 2. Overview of the proposed perceptual forgery fidelity assessment for Deepfake detection. The face quality mapping phase could distinguish real and fake faces more precisely by mapping them to varying fidelity scores based on their image quality. The feature extraction network is designed with proposed symmetric spatial attention augmentation to assess the perceptual fidelity of the face images.

Datasets

The authors evaluated their method on three major benchmarks:

  • FaceForensics++ (FF++): A standard dataset containing 1,000 real videos and 4,000 fakes generated by four different methods.
  • Celeb-DF (v2): A high-quality dataset with fewer visual artifacts, making it a tougher test for many detectors.
  • WildDeepfake: A challenging "in-the-wild" dataset collected from the internet, featuring diverse forgery methods, qualities, and backgrounds.

Key Insights and Ideas

  • Quality is a Forgery Clue: By explicitly modeling quality, the model becomes more robust. It learns that a lack of detail in a low-quality image isn't necessarily a sign of a "good" fake, but rather a lack of information.
  • Symmetry as a Structural Prior: Faces are naturally symmetric. Forgeries often break this symmetry in subtle ways (e.g., mismatched reflections in eyes). The SSAA module (shown below) bakes this domain knowledge into the network architecture, helping it spot these discrepancies more effectively than a standard vision Transformer.
Figure 3. Illustration of the proposed symmetric spatial attention augmentation (SSAA) performed on the feature maps in the shal- low layers.
  • Regression vs. Classification: Moving from classification to regression (predicting a continuous score) provides a "finer granularity" of data, which the authors show leads to better generalization on unseen datasets.

Key Results

DeepFidelity achieved top-tier results across the board:

  • Intra-dataset: It reached 98.89% accuracy on FF++ (HQ) and a perfect 100% accuracy/AUC on Celeb-DF (v2).
  • Real-world Performance: On the difficult WildDeepfake dataset, it achieved 88.34% accuracy, outperforming the previous state-of-the-art (SFDG) by nearly 4%.
  • Generalization: In cross-dataset tests (training on one dataset and testing on another), DeepFidelity consistently outperformed competitors, proving that "fidelity" is a more robust feature for detection than simple binary labels.

Limitations and Future Work

While successful, the authors note a few areas for improvement:

  • Quality Assessment Dependency: The framework currently relies on external face quality assessment models (like Ser-fiq). The authors suggest that developing quality assessment methods specifically tailored for deepfake detection could further boost performance.
  • Advanced Regression: Future work will explore more sophisticated regression models beyond the current SVR (Support Vector Regression) approach.
  • Unconditional Forgery: The current study focuses largely on face-swapping and reenactment; testing on "unconditional" forgeries (faces generated entirely from scratch by GANs or Diffusion models) remains a future direction.

Reference: Peng, C., et al. "DeepFidelity: Perceptual Forgery Fidelity Assessment for Deepfake Detection." arXiv:2312.04961v1, Dec 2023.

Summary updated

Abstract

Deepfake detection refers to detecting artificially generated or edited faces in images or videos, which plays an essential role in visual information security. Despite promising progress in recent years, Deepfake detection remains a challenging problem due to the complexity and variability of face forgery techniques. Existing Deepfake detection methods are often devoted to extracting features by designing sophisticated networks but ignore the influence of perceptual quality of faces. Considering the complexity of the quality distribution of both real and fake faces, we propose a novel Deepfake detection framework named DeepFidelity to adaptively distinguish real and fake faces with varying image quality by mining the perceptual forgery fidelity of face images. Specifically, we improve the model's ability to identify complex samples by mapping real and fake face data of different qualities to different scores to distinguish them in a more detailed way. In addition, we propose a network structure called Symmetric Spatial Attention Augmentation based vision Transformer (SSAAFormer), which uses the symmetry of face images to promote the network to model the geographic long-distance relationship at the shallow level and augment local features. Extensive experiments on multiple benchmark datasets demonstrate the superiority of the proposed method over state-of-the-art methods.
BibTeX
@article{peng2026deepfidelity,
  author = {Chunlei Peng and Huiqing Guo and Decheng Liu and Nannan Wang and Ruimin Hu and Xinbo Gao},
  title = {{DeepFidelity: Perceptual Forgery Fidelity Assessment for Deepfake Detection}},
  journal = {IEEE Trans. Circuits Syst. Video Technol.},
  year = {2026},
  doi = {10.1109/tcsvt.2025.3621207},
}
Figure