Motion-Locked Aggregation for Detecting Generated Video

Detectors for fake video keep going stale. Cues that single frames give away, such as blending seams, color mismatches and upsampling patterns, disappear as generators improve. Detectors that looked near-perfect on older benchmarks fall to near chance on cleaner ones, and to about 60% on real internet video. The field's answer has been to look across time instead, at motion trajectories, predictability and feature-space curvature. But temporal detectors have their own failure: in uncontrolled footage, camera shake, variable frame rates and compression swamp the temporal signal. Sequence models then do worse than per-frame models.

We borrow one design principle from a corner of chemistry. Some molecules glow only when crowded together, because crowding stops them twisting and so stops them losing their energy as heat. Other molecules glow alone and go dark when crowded, because neighbors drain their energy. The lesson is that aggregating many emitters helps only if you first freeze the motions that dissipate the signal; otherwise aggregation destroys it. Translated to video, we first fit and subtract the low-dimensional "nuisance" motion (camera motion, rigid and affine motion of each region). Only then do we pool over many tracked points. We then measure one specific thing: whether the leftover jitter at different points is correlated. Real footage should leave nearly independent tracking noise that averages away. Generated footage should leave a shared component that does not average away, because all points come from one latent sample that drifts from frame to frame.

Core Claim

After nuisance motion is removed, the residual motion of real and generated videos differs mainly in how its variance shrinks as more tracks are averaged, not in its size. For a set of $N$ tracked points with post-locking residuals $r_i(t)$, each with variance $\sigma^2$ and average pairwise correlation $\rho$, the variance of the pooled residual is

$$ \mathrm{Var}\Big[\tfrac{1}{N}\textstyle\sum_{i=1}^{N} r_i(t)\Big] \approx \frac{\sigma^2}{N} + \rho\,\sigma^2 . $$

This is the standard formula for the variance of a mean of equicorrelated variables. We predict $\rho\approx 0$ for real video after locking and $\rho>0$ for generated video. The claim has a second part: without locking, uncompensated camera motion inflates $\rho$ for real video too, so the two classes merge. This is our proposed explanation of the in-the-wild failure of 3D models reported in WildDeepfake. The new object is the correlation-versus-scale curve $\rho(N, \tau)$, estimated across spatial pooling sizes $N$ and temporal windows $\tau$ at several levels of locking. We use it as a detector feature and as a diagnostic.

Where the chemistry fits, and where it does not

The photophysics supplies a design principle, not equations. The rotor-free annulene system in Non-aromatic annulene-based aggregation-induced emission shows the principle cleanly. In solution a fast ring-flapping motion kills emission within about 4.9 ps. Packing into a solid blocks the flapping, the radiative lifetime extends to about 687 ps, and quantum yield rises from 0.7% to 10%. The macrocycle work in benzothiadiazole macrocyclization-induced emission enhancement shows that tying down conformational freedom and preventing close-contact quenching act together, raising solid-state yield from 65% to 99%. The quenching side, described in Grenz et al. and phenylethene-carbazole emitters, is the cautionary case: aggregation without constraint opens new loss channels.

Our analogy maps the dissipative molecular motions onto nuisance video motions and the loss channels onto camera and compression variance. The research file finds no prior work bridging these domains, and we do not claim the analogy is deeper than this. If the locking step and the $\rho$-scaling statistic do not beat simpler controls, the chemistry framing has earned nothing and we will say so.

Research Program

The pipeline has three stages. First, sparse Shi-Tomasi corners are tracked with Lucas-Kanade, following the robust tracking used in MotionPhys. Second, locking is applied at three nested levels, each fit robustly (RANSAC) per temporal window and subtracted from the trajectories: a global homography for camera motion, then a per-region affine model on superpixel or segmentation regions, then optionally a per-region smooth low-order polynomial in time for deliberate object motion. Third, for each locking level, we estimate $\rho$ over spatial pools of increasing size and over the 8-, 12- and 24-frame windows MotionPhys uses, plus longer ones. We also record the log-log slope of pooled variance against $N$. The slope is near $-1$ when residuals are independent and flattens toward 0 when a correlated floor dominates. The feature vector (the $\rho$ grid, slopes and residual spectra) feeds a small gradient-boosted classifier, matching MotionPhys's lightweight LightGBM setup so that any gain cannot be attributed to capacity.

As a secondary, diagnostic product we will compute "quenching curves" for existing detectors. These show how each detector's discriminative margin changes as scores are pooled over more frames and as videos are re-compressed. The prediction is that per-frame artifact detectors show the quenching pattern: their margin is high at one frame and falls under pooling and compression, consistent with Celeb-DF and VidProM. Locked-residual features should show the emission pattern: near zero per frame and rising with aggregation. If this holds, the curves give a cheap test of which cues are likely to go stale.

Baselines

The naive reference is a per-frame image classifier with scores averaged across the clip. We use XceptionNet, retrained on each training split with the same frame sampling as our method. Reported points are 69.25% accuracy on WildDeepfake (WildDeepfake) and a 50–60% average for eight image detectors on VidProM (VidProM). Beating it means a clear margin in-distribution and a large margin cross-generator.

The strongest trajectory-based competitor is MotionPhys. It computes curvature, angular velocity and jerk on unlocked 2D trajectories and reports 78.50% accuracy on GenBuster++ and 84.8% average AUC on AIGVDBench, trained on Open-Sora and tested on 30 other generators. It is the right comparison because it shares our input (sparse tracks) and classifier family, so any difference isolates the locking and correlation-scaling ingredients. We run it with the same tracker, the same track budget, the same LightGBM tuning budget and the same splits. Beating it means exceeding 84.8% average AUC on the AIGVDBench cross-generator protocol. No reported result found on Lacuna for MotionPhys on WildDeepfake, so we will run it there ourselves.

The strongest method on text-to-video benchmarks is ReStraV, which measures trajectory curvature in frozen DINOv2 features. It reports 97.17% accuracy and 98.63% AUROC on VidProM. We do not expect to beat it in-distribution there. The comparison that matters is cross-generator transfer on AIGVDBench and robustness on WildDeepfake. We will also test whether concatenating our features with ReStraV's improves on ReStraV alone. The compressed-domain method STREAM reports 0.965 average mAP across VidProM and four other sets. We will compare against it on H.264 and after re-encoding to HEVC and AV1. Its motion-vector chains are expected to break under re-encoding, whereas our tracks come from decoded frames.

The key ablations are MotionPhys features computed on locked trajectories, and our $\rho$ features computed without locking. The first isolates whether correlation scaling adds anything beyond stabilization. The second tests whether locking is what makes aggregation help.

Why not just…?

Why not just stabilize the video and run MotionPhys? This is the most serious threat, and it is exactly the first ablation. Stabilization removes camera motion but leaves MotionPhys measuring per-track shape (curvature, jerk). Our claim is that the discriminative information lies in cross-track correlation, which per-track descriptors average over. If stabilized MotionPhys matches the full method within noise on AIGVDBench and WildDeepfake, the correlation statistic is redundant. The contribution would then shrink to "stabilize before measuring kinematics", and we would report it that way.

Why not just train a large end-to-end video network? Fixed spatio-temporal kernels are reported to fall behind 2D models on internet video (WildDeepfake). Gains from region-adaptive weighting (Region-Aware Temporal Inconsistency Learning, 93.31% on DFDC) suggest nuisance and forgery cues need explicit separation. Learned models also overfit generator signatures (DNA). Our cross-generator protocol would expose that failure if it occurs.

Why not just use ReStraV, which already scores near ceiling? Its authors note that curvature in a known feature space can be optimized away by future generators, and that it ignores time direction. Our statistic lives in pixel-track space and targets a property of shared latent sampling. It could also be regularized against, but that would be a different and plausibly costlier constraint for a generator. We test complementarity rather than claim replacement.

Decisive Checks

Three results would settle the claim. First, on WildDeepfake and Celeb-DF real videos, the estimated $\rho$ should fall toward zero as locking level increases, while generated videos keep a positive floor. Celeb-DF's Kalman temporal smoothing is a useful stress test, because it suppresses jitter but should not remove shared drift. Second, on AIGVDBench trained only on Open-Sora, the full method should exceed MotionPhys's 84.8% average AUC and outperform the stabilized-MotionPhys ablation. Third, performance should hold after HEVC and AV1 re-encoding, where STREAM's inputs degrade. If the first check fails, meaning real and generated videos have similar $\rho$ after locking, the central hypothesis is wrong and the project reduces to an evaluation of stabilization.