Does the Texture Move With the Skin? Canonical-Persistence Forensics for Video Deepfakes

Surgical vision has spent years learning to explain video of soft, moving tissue as one persistent 3D surface that deforms over time. Systems built on Gaussian splatting represent a scene as a cloud of small colored 3D blobs that are rendered by projecting them onto the image. These systems fit a short clip of a stretching organ in minutes by keeping each blob's color fixed and letting only its position move. We propose to turn that fitting procedure into a deepfake detector. A real face, like a real liver, is one piece of skin whose texture is carried along by its motion. A face-swapped or generated face is painted anew in every frame, so its texture drifts relative to any consistent underlying surface. If we fit a deforming 3D model to a clip twice, once forcing each blob's color to stay fixed and once allowing per-frame color changes, the gap between the two fits measures how much of the video cannot be explained as "the same surface, moving." We expect that gap to be small for real video and large for forgeries, whichever generator produced them.

This matters because deepfake detectors fail when they meet generators they were not trained on. They latch onto generator-specific pixel fingerprints that new synthesis methods do not share and that video compression erases. A test of whether texture persists on a moving surface does not depend on any particular generator's fingerprint, and it is computed over dozens of frames rather than from fine pixel detail. That gives a concrete reason to expect it to survive both a change of generator and heavy compression.

Core Claim

The obvious bridge between these fields is to fit a 4D deformation model to face video and flag "physically implausible motion": strain, jerk, volume change. That idea has a flaw the literature makes visible. In the dominant forgery type, face swapping, the motion comes from a real driving video, so the kinematics are genuine and only the texture is synthesized. Our claim is that the useful signal is not whether the motion is plausible but whether appearance stays attached to the deforming surface. We call this canonical persistence: the clip can be explained as one fixed "canonical" set of colored blobs that is moved and stretched over time.

The proposal treats tissue deformation modeling as the mechanism and cross-dataset generalization as the regime where the mechanism should pay off. It makes a falsifiable prediction about where each signal works. The persistence gap should carry face-swap detection, where motion is real. Kinematic statistics should add value mainly for reenactment and talking-face generators, which synthesize the motion itself.

Concretely, for a clip $V$ we fit the same deformable Gaussian model under two appearance regimes and define the persistence gap

$$G(V) = \frac{\mathcal{L}_{\text{static}}(V) - \mathcal{L}_{\text{free}}(V)}{\mathcal{L}_{\text{free}}(V)},$$

where $\mathcal{L}_{\text{static}}$ is the converged photometric error when each Gaussian's color is fixed across time (modulated only by a low-dimensional per-frame global lighting term shared by all Gaussians), and $\mathcal{L}_{\text{free}}$ is the error when each Gaussian also receives a small per-frame color offset. Normalizing by $\mathcal{L}_{\text{free}}$ removes clip-level differences in sharpness and compression.

Relation to Prior Work

The fitting machinery comes directly from surgical reconstruction. Efficient Deformable Tissue Reconstruction via Orthogonal Neural Plane factorizes space-time into static spatial planes and dynamic space-time planes, which is known as a HexPlane. EndoGaussian attaches such a HexPlane plus a tiny decoder to 3D Gaussians initialized from monocular depth. It fits ENDONERF clips in under two minutes on 2GB of GPU memory. Endo-4DGS adds confidence weighting and normal constraints to cope with unreliable monocular depth, and HFGS adds flow-guided temporal supervision for fast motion.

We reuse their design, with one inversion. EndoGaussian and EndoMD-SLAM list static appearance as a limitation, because real tissue has moving specularities. We use that same constraint as the measurement instrument. A small shared lighting term, in the spirit of Forplane's handling of moving light sources, absorbs global illumination changes, so that only local, surface-detached appearance change is penalized.

On the forensics side, three lines of work motivate the proposal. FInfer shows that temporal unpredictability exposes high-quality fakes. It reaches 70.60% AUC from FaceForensics++ to Celeb-DF, but it predicts in a 128-dimensional 2D latent space that mixes head motion, identity and texture. Recurrent Convolutional Strategies and Thumbnail Layout learn flicker as black-box 2D patterns. Our score is instead a residual of an explicit geometric model that separates rigid head motion from non-rigid deformation and both from appearance.

Implicit Identity Leakage shows that detectors learn identity shortcuts. That constrains our design: we feed the classifier a few dozen scalar statistics of the fit, not images, so identity has little room to leak. A similar rationale underlies the receptive-field truncation of What Makes Fake Images Detectable?, which limits capacity to block shortcuts.

The closest papers bridging the two fields, ImplicitDeepfake and A Controllable 3D Deepfake Generation Framework with Gaussian Splatting, use Gaussian splatting to generate forgeries, not to detect them. They also pose a direct threat, discussed below. A generator built on a persistent 3D representation might pass our test by construction.

Research Program

We first build a face-clip fitter. It takes 48-frame face crops at 256 pixels, initializes Gaussians from monocular depth as in EndoGaussian, and estimates a per-frame rigid head pose jointly with the HexPlane deformation, following the joint pose and deformation optimization of FLex. We run the static-color and free-color fits from a shared warm start, which roughly doubles the cost of a single fit.

From each clip we extract two groups of features. The first is the persistence gap $G$, computed globally and per facial region (inner face versus hairline and jaw), since swaps typically leave the periphery real. The second group is kinematic statistics of the fitted deformation field: mean local area change between neighboring Gaussians, temporal jerk of their trajectories, and the fraction of Gaussians whose neighbors change over time.

A logistic regression or a two-layer network trained on FaceForensics++ maps these features to a label. We also report the training-free score $G$ alone, which should transfer if the hypothesis is right.

Before any forensic claim, we validate the fitter on real non-rigid motion. On ENDONERF, our static-color fit should land near EndoGaussian's reported 37.849 dB PSNR. That check confirms the implementation. It also gives a hard negative control: real tissue undergoing large deformation, including cutting and tearing. Tearing changes the surface topology and should raise $G$, so ENDONERF shows how much false alarm a genuine but violent deformation produces. That in turn tells us how badly mouth opening and tongue exposure in real talking faces will hurt.

Baselines

The naive reference point is a full-depth XceptionNet binary classifier trained on FaceForensics++ HQ crops, as in FaceForensics++. It reports 99.26% accuracy on raw and 81.00% on low quality in-domain. Implicit Identity Leakage reports that a plain binary baseline reaches roughly 48% AUC on DFDC-V2. This reference shows the size of the gap we are trying to close.

The strongest existing image-level competitor is the ID-unaware model of Implicit Identity Leakage. Trained on FaceForensics++, it reports 93.88% AUC on Celeb-DF and 73.85% on DFDC-V2. It is the strongest because it already removes the identity shortcut that otherwise dominates cross-dataset failure.

We do not expect to beat 93.88% on Celeb-DF v2 with a standalone persistence score. Celeb-DF's swaps are frame-wise and the existing method is already strong there. The claim we stake instead has three parts. First, we should exceed 73.85% on DFDC-V2. Second, we should exceed the 73.8% average video-level AUC from FaceForensics++ to Celeb-DF++ reported by Celeb-DF++. Third, we should lose less than that paper's reported 7.5% average drop under c45 compression. A fusion of our features with the ID-unaware score should improve on the ID-unaware model alone in every cross-dataset setting. If fusion adds nothing, the signal is redundant.

The strongest temporal competitor is FInfer, with 70.60% cross-dataset AUC from FaceForensics++ to Celeb-DF. We should beat it on that exact protocol. We also run the truncated patch classifier of What Makes Fake Images Detectable?. No reported result was found on Lacuna for it on these video benchmarks, so we will report our own run.

All learned baselines use the same FaceForensics++ HQ training videos, the same face crops and the same frames per video. Each gets the same hyperparameter search budget, selected only on a FaceForensics++ validation split, never on target data.

Three ablations isolate the key ingredient. The first is kinematics only: deformation statistics without $G$. The second is a 2D flow-warp persistence score, described next. The third swaps our Gaussian surface for an off-the-shelf parametric face-model fit and uses its texture residual.

Why not just…?

Why not just warp frames with optical flow and measure the residual? This is the most serious threat. Warping frame $t$ onto frame $t+k$ with estimated flow and averaging the photometric error is a cheap 2D version of persistence, and it may capture most of the signal. Our argument for the 3D model is that pairwise flow cannot enforce persistence over a whole clip. Pairwise flow loses track of skin that rotates out of view and back. It also absorbs slow texture drift into the flow itself, because the error is spread over many small steps. A single canonical surface cannot hide drift that way.

We test this directly. We stratify Celeb-DF++ clips by estimated head rotation and compare the two scores in each stratum. If flow-warp matches our score even at large rotations, we will report that and conclude that the explicit 4D model is unnecessary.

Why not just use a 2D temporal model such as FInfer? FInfer's 70.60% cross-dataset AUC shows that learned temporal prediction transfers only partly. Its latent space mixes head motion and identity with forgery cues, which is the entanglement Implicit Identity Leakage warns about. The head-to-head on the same protocol settles whether explicit separation of motion from appearance helps.

Why not just fit a parametric face model and look at its residual? Many reenactment and talking-face generators are driven by keypoint or face-model motion. Their geometry is therefore plausible under such a model by construction, while persistence of skin texture is not enforced. A face model with a learned texture basis can also re-explain swapped texture as a valid identity. We include this as an ablation. If it matches us, the surgical machinery adds only flexibility for hair, occlusion and non-face regions, and we will say so.

Why not just fine-tune a large pretrained encoder on more varied forgeries? Celeb-DF++ reports that recent high-capacity detectors still degrade sharply across face-swap, reenactment and talking-face scenarios, and that adding data without diversity does not fix this. Our features are complementary rather than competing, which the fusion experiment tests.

Won't a 3D-Gaussian generator defeat this? Possibly, and this is the honest limit. Forgeries produced like ImplicitDeepfake render one persistent representation and should show a small $G$. We will generate a small set of such forgeries as a red-team test. Our prediction is that only the kinematic features retain any power there.

Proposed Experiments and Decisive Checks

Training uses FaceForensics++ HQ. Evaluation uses Celeb-DF v2, DFDC-V2, and the three protocols of Celeb-DF++ (ouc-vas/celeb-df-pp): cross-scenario (GF-eval), compression down to c45 (GFQ-eval) and cross-dataset (GFD-eval).

Fitting cost is the main practical constraint. At EndoGaussian's reported speed of under two minutes per fit, every Celeb-DF++ video is out of reach. We therefore evaluate on a stratified subset of about 3,000 videos covering all 22 methods and all real videos, and we report confidence intervals. In parallel, we test whether a 16-frame, 128-pixel fit preserves the ranking of $G$, which would make full-scale evaluation feasible.

Three results decide the proposal. The first is the mechanism check. On FaceForensics++ swap methods (DeepFakes, FaceSwap), $G$ alone should separate real from fake well above chance while kinematic statistics alone stay near chance. On Celeb-DF++ talking-face methods, kinematic statistics should contribute measurably. If the pattern is reversed, or both are flat, the hypothesis is wrong.

The second is the generalization check. The learned score should exceed 73.8% average video-level AUC on Celeb-DF++ and 73.85% on DFDC-V2, and its drop at c45 should be smaller than 7.5%.

The third is the necessity check. The 4D score should beat the flow-warp ablation in the large-rotation stratum, and real ENDONERF clips without tearing should score in the range of real FaceForensics++ faces. A large false-alarm rate on that genuine non-rigid motion would mean $G$ is measuring deformation difficulty rather than forgery.