A dual-stream model based on PRNU and quaternion RGB for detecting fake faces

Man Hua, Shuangshuang Li, Jinwei WangPLoS 2025 · DOI

TL;DR

This paper introduces a novel dual-stream deep learning model for detecting AI-generated (AIGC) fake faces. By combining Photo Response Non-Uniformity (PRNU) for global features and quaternion RGB for local features, the model achieves a remarkable 96.81% accuracy in distinguishing real from fake faces, addressing the limitations of existing methods that often focus on only one type of manipulation.

What Problem Does This Paper Address?

The proliferation of AI-generated content (AIGC), particularly realistic fake faces, presents a significant challenge for digital forensics and societal security. Current methods often struggle to detect both globally manipulated faces (e.g., generated by GANs like StyleGAN) and locally manipulated faces (e.g., face-swapping via Deepfake) effectively. This lack of a universally applicable detection methodology means that existing models are often not robust enough to handle the diverse and sophisticated forgery techniques emerging today, leading to a critical need for more comprehensive and adaptable fake face detection systems.

Methods

The core of this work is a dual-stream CNN architecture designed to capture both global and local facial manipulation artifacts.

  • PRNU Stream: This stream focuses on global features. It leverages Photo Response Non-Uniformity (PRNU), often referred to as a "camera fingerprint," which captures subtle, sensor-level noise patterns. The paper uses a Haar wavelet transform for PRNU extraction, which is noted for its ability to decompose images multiscale and capture high-frequency noise, thus better characterizing global facial manipulation. This stream is trained end-to-end to learn complex PRNU feature mappings.
  • Quaternion RGB Stream: This stream targets local manipulation. It utilizes quaternion neural networks (QNNs). Quaternions, a type of hypercomplex number, allow for a more integrated processing of RGB color channels, capturing inter-channel dependencies and spatial structures that traditional CNNs might miss. This is particularly useful for detecting subtle color inconsistencies, background anomalies, and spatial distortions introduced by local tampering. The paper details quaternion convolution and max-pooling layers designed to handle this multi-dimensional data effectively.
  • Feature Fusion: The outputs from both streams are fused to combine their complementary strengths, allowing the model to leverage both global (PRNU) and local (quaternion RGB) information for a more robust and accurate detection.

Datasets

The study utilizes the following datasets:

  • FFHQ (Flickr-Faces-HQ): Used for training and evaluation, this dataset comprises 70,000 facial images with diverse demographics and poses.
  • Celeb-DF-V1: A publicly available dataset commonly used for deep learning in facial forgery detection and deepfake recognition.
  • StyleGAN-generated faces: Used for training and evaluation.
  • Celeb-DF-V1-generated faces: Used for training and evaluation.

The paper standardizes image dimensions to 256x256 for consistent processing. For ablation studies and comparisons, a subset of 5000 real faces (from FFHQ and Celeb-DF-V1) and 5000 fake faces (generated by StyleGAN and Celeb-DF-V1) were used for training. The final evaluation was performed on a test set of 2000 real and 2000 fake faces.

Key Insights and Ideas

  • Complementarity of Global and Local Features: The most significant insight is that effectively detecting diverse fake faces requires a two-pronged approach. PRNU captures the "camera fingerprint" for global inconsistencies, while quaternion RGB excels at discerning subtle local artifacts. Neither alone is sufficient, but their fusion creates a powerful detector.
  • Quaternion Networks for Color Channel Integration: Quaternion algebra provides a natural framework for processing multi-channel data like RGB images. This allows QNNs to capture inter-channel correlations more effectively than standard CNNs, which is crucial for detecting subtle color shifts and inconsistencies in manipulated regions.
  • Haar Wavelets for PRNU: Using Haar wavelets for PRNU extraction is highlighted as an improvement over traditional methods, enabling better capture of multiscale features and high-frequency noise, which are indicative of global manipulation.
  • End-to-End PRNU Learning: Moving PRNU feature extraction into a deep learning pipeline (end-to-end learning) allows the network to automatically learn the most discriminative PRNU features, overcoming limitations of manually designed feature extraction.

Key Results

  • High Accuracy: The proposed dual-stream model achieved a remarkable 96.81% detection accuracy on diverse datasets.
  • Ablation Study Success: Individual streams (PRNU or Quaternion RGB) performed below 95% accuracy, underscoring the necessity of the dual-stream approach. The PRNU stream with Haar wavelet extraction showed significant improvement over conventional noise reduction methods.
  • Quaternion vs. Other Hypercomplex Numbers: Quaternion CNNs outperformed traditional CNNs and showed better results than octonion or sedenion CNNs for this task, suggesting an optimal balance in dimensionality for capturing local features without introducing excessive complexity or noise.
  • Network Depth Optimization: A 4-layer network structure was found to be optimal, balancing performance with computational efficiency.
  • Generalization: The model demonstrated satisfactory performance across multiple fake face datasets (StyleGAN II, Celeb-DF-V2, StyleGAN III, FaceForensics++, BigGAN, DFDC), indicating good generalization capabilities.
  • Speed: While not the absolute fastest, the model offers a good balance between accuracy and real-time performance, making it suitable for practical applications.

Limitations and Future Work

  • Static Images Only: The current model is designed for static images. The authors acknowledge that detecting fake facial videos requires considering dynamic information like inter-frame continuity and motion trajectories, which is a significant future research direction.
  • Real-time Performance: While the model offers a good balance, it is not the fastest method. Further optimization for real-time applications, especially in scenarios like social media content review, is a potential area for improvement.
  • Hypercomplex Number Exploration: The paper briefly touches upon octonion and sedenion, suggesting that exploring other hypercomplex numbers might yield further improvements, though with increased computational cost.

Summary updated

Abstract

The forensic examination of AIGC(Artificial Intelligence Generated Content) faces poses a contemporary challenge within the realm of color image forensics. A myriad of artificially generated faces by AIGC encompasses both global and local manipulations. While there has been noteworthy progress in the forensic scrutiny of fake faces, current research primarily focuses on the isolated detection of globally and locally manipulated fake faces, thus lacking a universally effective detection methodology. To address this limitation, we propose a sophisticated forensic model that incorporates a dual-stream framework comprising quaternion RGB and PRNU(Photo Response Non-Uniformity). The PRNU stream extracts the "camera fingerprint" feature by discerning the non-uniform response of the image sensor under varying lighting conditions, thereby encapsulating the overall distribution characteristics of globally manipulated faces. The quaternion RGB stream leverages the inherent nonlinear properties of quaternions and their informative representation capabilities to accurately describe changes in image color, background, and spatial structure, facilitating the meticulous capture of nuanced local distinctions between locally manipulated faces and real faces. Ultimately, we integrate the two streams to establish the exchange of feature information between PRNU and quaternion RGB streams. This strategic integration fully exploits the complementarity between two streams to amalgamate local and global features effectively. Experimental results obtained from diverse datasets underscore the advantages of our method in terms of accuracy, achieving a detection accuracy of 96.81%.
BibTeX
@inproceedings{hua2025dual,
  author = {Man Hua and Shuangshuang Li and Jinwei Wang},
  title = {{A dual-stream model based on PRNU and quaternion RGB for detecting fake faces}},
  booktitle = {PLoS},
  year = {2025},
  doi = {10.1371/journal.pone.0314041},
}
Figure