Abstract
The rapid advancement of deepfake generation techniques poses significant security and privacy risks, particularly in video conferencing scenarios where variable resolutions, compression artifacts, and environmental factors complicate detection. Existing benchmarks often fail to address these context-specific challenges, limiting their applicability to real-world communication platforms. To bridge this gap, we introduce the VCF (Video Conference DeepFakes) dataset, the first specialized benchmark designed for evaluating deepfake detection in video conferencing contexts. VCF leverages the VCD dataset as target videos and the LaPa dataset as a set of source faces, algorithmically ranking sources by similarity in gender, ethnicity, age, and facial hair to select optimal matches for enhanced deepfake visual plausibility. The dataset incorporates multi-resolution videos, H.264 compression artifacts from different compression rates, and diverse backgrounds to simulate conditions specific to video conferences. Comprehensive evaluations of 14 detection methods reveal significant performance degradation under video quality variations. Our results emphasize the critical need for robust detection frameworks resilient to resolution shifts, compression artifacts, and diverse generation pipelines. VCF provides a standardized, scenario-specific benchmark to drive advancements in securing digital communication platforms against evolving deepfake threats.
Introduction
Most deepfake detection benchmarks focus on high-quality, general-purpose video manipulations. However, video conferencing presents a unique set of challenges that these benchmarks often ignore. Network conditions frequently force streams into low resolutions, and standard codecs like H.264 introduce compression artifacts that can mask manipulation signatures. Furthermore, real-world attackers are unlikely to swap faces at random; they choose source faces that match the target's demographic attributes to minimize visual glitches. The VCF dataset addresses the lack of a "stress test" for detectors in these realistic, user-centric scenarios.
The VCF Generation Pipeline
The researchers built the VCF dataset using a systematic pipeline designed to emulate a sophisticated attacker. The process involves four key stages:
- Source & Target Selection: Authentic target videos were sourced from the Video Conference Dataset (VCD), while source faces were drawn from the LaPa dataset.
- Face Matching Protocol: Instead of random swaps, a multivariate similarity metric matched source and target faces based on gender, ethnicity, age, and facial hair. This ensures the resulting deepfake is visually plausible and requires fewer transformations.
- Generation: Two popular open-source frameworks were employed: SimSwap (for high-fidelity identity transfer) and Deep-Live-Cam (for real-time simulation).
- Post-Processing: To simulate actual call conditions, videos were downsampled to multiple resolutions (including 1080p, 540p, and 270p) and subjected to H.264 compression with varying Quantization Parameters (QP 23 for high quality, QP 40 for low quality).
Dataset Characteristics
The VCF dataset consists of 1,920 authentic video clips and 7,680 deepfake variants. When compared against established benchmarks like FaceForensics++ (FF++) or Celeb-DF, VCF is unique in its explicit focus on the video conferencing setup and its inclusion of multi-resolution and multi-quality tiers. This structure allows researchers to evaluate how detection models handle the specific perturbations common in digital communication.
Results and Robustness Evaluation
An evaluation of 14 competitive detectors using video-level AUC (Area Under the Curve) revealed significant vulnerabilities in current state-of-the-art models.
- Resolution Sensitivity: Detection accuracy is not constant across quality tiers. Models that perform excellently on 1080p video see their performance drop sharply when resolution decreases. This suggests that current detectors rely heavily on high-frequency details that are lost during downsampling.
- Compression as a Shield: High H.264 compression (QP 40) acts as a natural "adversarial attack," smoothing out the artifacts that detectors typically identify.
- Performance Gaps: Even the top-performing model, X-CLIP, saw its AUC drop from 0.804 on "1080p, raw" videos to just 0.560 on "270p, c40" (low resolution, high compression) videos. For nearly all realistic conferencing subsets, the ROC-AUC did not exceed 0.7, a level considered insufficient for reliable real-world deployment.
Conclusion
The VCF benchmark serves as a call to action for the research community. The significant performance degradation observed under variable resolutions and compression rates proves that generalization remains an unsolved challenge. Future work must focus on developing detection frameworks that are inherently resilient to the environmental and technical noise typical of real-world video conferencing to effectively secure digital communication platforms.
Summary updated