Early deepfake detectors operated like digital microscopes, hunting for "blending ghosts," checkerboard artifacts, or high-frequency noise left behind by generative models. However, these low-level pixel artifacts are fragile. When a video is uploaded to social media, heavy H.264 compression and resizing act as a "laundry" process, scrubbing away the fine-grained textures that traditional detectors rely on.
To solve this, researchers have shifted the focus from how an image was generated to who is in the image and how they move. By prioritizing high-level semantic features—such as 3D bone structure, identity consistency, and behavioral dynamics—modern forensic tools can identify forgeries even in low-quality, heavily compressed media.
The Fragility of Pixels vs. the Stability of Semantics
The core intuition behind this research direction is that while a generative model can easily fool a pixel-level check, it struggles to maintain the complex, physical realities of a human subject.
Hanqing Zhao and Wenbo Zhou demonstrate this in their Self-supervised Transformer for Deepfake Detection by freezing the early layers of their model. This forces the network to ignore "brittle" low-level artifacts and instead learn high-level semantic representations. These semantic cues—like the way a mouth shapes a specific phoneme—are structural. They survive compression because they are fundamental to the scene's meaning, not just noise in the signal.
Identity as a Forensic Signal
One of the most powerful semantic signals is identity consistency. In a face-swap, a "mask" of a target person is placed over the "base" of a source person. This often creates a fundamental mismatch between the inner face and the rest of the head.
Xiaoyi Dong and Dongdong Chen pioneered this in Identity-Driven DeepFake Detection. Their "OuterFace" model ignores the easily manipulated inner face and focuses on the hair, ears, and head shape. If the identity suggested by the inner face doesn't match the biometric "fingerprint" of the outer face, the video is flagged. Similarly, Baojin Huang and Zhongyuan Wang use Implicit Identity Driven Deepfake Face Swapping Detection to compare the "explicit" appearance of a face against the "implicit" underlying identity, spotting forgeries where the two diverge.
Anatomical and Geometric Constraints
Beyond identity, the physical laws of the human body provide a "ground truth" that deepfakes often violate.
- 3D Structure: Wei Wang and Jing Dong use 3D Morphable Models (3DMM) in their Robust Face-Swap Detection to show that while a face might look like a celebrity, the underlying "bone structure" often belongs to the original actor.
- Anatomical Coupling: Shruti Agarwal and Hany Farid found that deepfake creators often overlook the ears. In Detecting Deep-Fake Videos From Aural and Oral Dynamics, they demonstrate that the subtle movements of the ear are physically coupled with the jaw. Deepfake generators, which usually focus only on the mouth and eyes, fail to replicate this mechanical relationship.
- Geometric Trajectories: Zekun Sun and Na Ruan developed LRNet, which tracks the trajectories of facial landmarks. By smoothing out "jittery" data, they can detect when the geometric movement of a face feels unnatural, a signal that remains detectable even in low-bandwidth environments.
Behavioral Dynamics and Speaking Signatures
The final frontier of this direction is the "speaking signature"—the unique way an individual coordinates their head, eyes, and mouth.
Alexandros Haliassos and Stavros Petridis introduced RealForensics, which learns the "language" of natural facial movements through self-supervised audio-visual alignment. By understanding how a real person's lips move in sync with their voice, the model can spot the subtle "identity jitters" or temporal inconsistencies common in deepfakes, as also explored by Baoping Liu in TI2Net.
This culminates in "Person-of-Interest" (POI) forensics. Davide Cozzolino and Luisa Verdoliva argue in POI-Forensics that we shouldn't just ask "is this a deepfake?" but "is this person acting like themselves?" By building a unique audio-visual signature for a specific person, detectors can spot deviations in vocal characteristics and head poses (Dingyu Lu et al., STT) that are nearly impossible for current generative models to mimic perfectly.