Abstract
As deepfake generation techniques evolve to produce visually indistinguishable content, traditional artifact-based detection methods are becoming less reliable. This paper introduces a "social verification" framework that leverages multi-view video captures of the same event. By analyzing the geometric consistency of facial landmarks—specifically mouth movements—across different camera angles, the system can "crowd-sign" authentic footage and isolate digitally manipulated outliers. Our approach achieves 75% detection accuracy in multi-camera scenarios and operates with high computational efficiency.
Introduction: The Limits of Artifact Hunting
The "arms race" between deepfake generators and detectors is reaching a tipping point. Most current detectors look for visual glitches—like unnatural blinking or temporal flickering—but as generative models (GANs) improve, these "tells" are vanishing. We can no longer rely on a single video to prove its own truth.
This research proposes a shift toward a future where multiple "citizen journalists" or news organizations record the same event. In such an "untrusted" environment, the challenge is to reach a consensus on which video is real if an attacker injects a visually perfect deepfake into the stream. The solution lies in finding a "visual hash": a representation of the scene that remains consistent across all real cameras regardless of angle, but breaks when a face is digitally manipulated.
Methodology: Face Geometry Consistency
The researchers focus on Face Geometry Consistency as the core verification metric. The pipeline operates through several distinct stages:
- Landmark Fitting: 2D facial landmarks are extracted from every frame of synchronized video feeds using the Face Alignment Network.
- Parts-based Normalization: The system isolates specific regions, primarily the mouth, as it is the most frequent target for deepfake manipulation (e.g., altering speech).
- Landmark PCA: To create a compact motion signal, Principal Component Analysis (PCA) is performed on the landmarks. The face's pose at each frame is represented as the Mahalanobis distance from the center of the PCA space, creating a time-series signal that captures the variance of mouth motion.
- Hierarchical Clustering: These signals are compared across all cameras. Because real cameras observe the same physical mouth, their geometric signals should cluster together. A deepfake—even a visually perfect one—will exhibit slightly different geometric "rhythms" because it was generated by a model rather than physical reality.
- Social Verification: The system uses a "majority rule" to sign the videos that agree with each other as authentic and flag outliers as fakes.
Key Insights and Results
- Geometry as a Visual Hash: The core innovation is treating facial geometry as a view-invariant hash. While the visual image changes based on camera angle, the underlying motion geometry remains consistent across all honest observers.
- Superiority of 2D Landmarks: Surprisingly, fitting complex 3D face models (like 3DMM) was less effective than the 2D landmark variance method. 3D models tended to "smooth over" the subtle geometric errors that make a deepfake detectable.
- Detection Performance: The system achieved 75% accuracy in detecting a single fake out of six videos using an 8-second window. Accuracy decays gracefully as the number of "colluding" fakes increases.
- Real-time Viability: The implementation is highly efficient, running in approximately 0.01 seconds per 250-frame window on a standard CPU.
Limitations and Future Outlook
While promising, the method faces challenges when the speaker is far from the camera, as landmark noise can overwhelm the motion signal. Additionally, the system currently processes time windows independently; temporal propagation of detections could further boost accuracy.
A critical vulnerability remains the "Deepfake Flashmob": if an attacker controls the majority of cameras in a space and feeds them the same fake, the system would incorrectly verify the fake as truth. Future work will likely involve integrating cross-camera audio analysis to create a more robust, multi-modal verification suite.
Summary updated