Secret Spot-Checks of Lighting for Deepfake Detection on Unseen Datasets

Deepfake detectors learn from one dataset and then fail on the next. A detector trained on FaceForensics++ typically falls to near-chance accuracy on Celeb-DF, because it has memorized traces left by particular generators rather than anything that makes an image fake in general. Physical lighting is a natural candidate for something more general. Light in a real scene comes from the same sources everywhere, so a pasted face, a swapped identity, or a harmonized splice must keep shading and highlights consistent across regions it never saw. Lighting checks have one weakness that matters in practice. They are fixed and public, so a forger who knows which regions get compared can repair exactly those. This proposal pairs lighting-consistency checks, trained only on real photographs, with a secret key that decides, image by image, which region pairs and which lighting tests get evaluated. A forger who does not hold the key cannot know which comparisons will be made, so the only reliable way to evade every possible check is to make the whole image's lighting physically consistent.

The key is managed with a puncturable pseudorandom function: a keyed scrambling function whose key can be handed out with a small set of inputs "punched out," so the holder can compute every output except those. This lets the auditor hand verifiers a key that reproduces all public checks while a reserved set of checks stays hidden, even if the verifier's software leaks. The cryptographic piece is deliberately small. The scientific bets are that real-only lighting consistency narrows the cross-dataset gap, and that secret selection of checks makes it robust to forgers who adapt to the detector. We test both directly, and we include an ablation in which the puncturable function is replaced by two ordinary independent keys.

Core Claim

Lighting consistency, scored as agreement between pairs of image regions and trained without any fake examples, generalizes across datasets better than artifact-based classifiers. Its weakness to adaptive forgers comes from public knowledge of which comparisons are made, not from the physics. Choosing those comparisons with a secret key should preserve most of the clean-data detection power under white-box attack on the public checker. Puncturing should then let that secrecy survive delegation of the checker to untrusted parties. What is new is the measured claim that keyed, partially withheld lighting checks resist adaptation and transfer across datasets. Neither a new lighting estimator nor a new cryptographic primitive is being proposed.

Where This Sits

Lighting forensics already exists as a set of narrow, fixed tests. Cast-shadow rays from several objects must meet near a common light source, the "wedge" constraint surveyed in Visual Computing and Machine Learning Techniques for Digital Forensics. Combining Color and Geometry for the Active, Visual Recognition of Shadows bounds whether an object could cast a given shadow. Algorithms for verification of space images by shooting options checks satellite shadows against solar metadata, which illustrates the constraint that metadata is often stripped, so our checks use pixels only.

Two papers motivate us most directly. Exposing GAN-Generated Faces Using Inconsistent Corneal Specular Highlights reaches AUC 0.94 on FFHQ versus StyleGAN2 by comparing reflections in the two eyes, but only on frontal, high-resolution portraits. Disharmony: Forensics using Reverse Lighting Harmonization localizes harmonized composites at ROC AUC ≈0.70, where noise-based detectors score 0.52–0.63. It also produces false positives under global diffusion edits and misses small edits. Both papers use one public test, and neither is evaluated on the face-deepfake cross-dataset benchmarks. We turn their single fixed comparison into a sampled family of comparisons and evaluate on FF++, Celeb-DF and DeeperForensics. Inharmonious Region Localization via Recurrent Self-Reasoning contributes a usable method idea: compare each region with averaged background statistics. We reuse that region-versus-context structure.

On the generalization side, Towards Generalizable Deepfake Detection with Locality-aware AutoEncoder and On Improving Cross-dataset Generalization of Deepfake Detectors diagnose overfitting to generator traces. The latter reaches AUC 0.669 on Celeb-DF by having a learned policy choose brightness and contrast augmentations at test time. What Makes Fake Images Detectable? Understanding Properties that Generalize shows that small-receptive-field patch classifiers transfer across GANs. All three still learn from fake examples. Our scorer never does, which is the reason to expect it to transfer.

Proactive approaches such as All in One: Unifying Deepfake Detection, Tampering Localization, and Source Tracing with a Robust Landmark-Identity Watermark reach AUC 0.9388 by embedding landmarks before release. They cannot help with legacy media. Watermarking Cryptographic Capabilities constrains our design: functions that can be learned from queries cannot carry durable watermarks. We therefore place no secret in the image at all. The secret lives in the verifier's choice of tests. The puncturable functions of Boyle et al. and Zhandry (Boyle et al., Zhandry) supply the key-delegation method. We need only the basic, non-private version, which can be built from a tree of ordinary pseudorandom generator calls. The heavy machinery behind private constraints (Privately Constrained PRFs from DCR: Puncturing and Bounded Waring Rank) is not required.

Research Program

Lighting estimator. For each image region, a monocular surface-normal predictor plus a shading decomposition produces a low-order lighting estimate. This is second-order spherical harmonics, nine numbers describing how much light arrives from each direction. Where eyes are visible, it also produces a corneal-highlight shape.

Consistency scorer. A small network scores a pair of regions for lighting consistency. It is trained contrastively on real photographs only: pairs from the same photo count as consistent, and pairs re-lit from different photos count as inconsistent. We build the latter with Careaga-style physics composites and with relighting augmentation, never with deepfake generators.

Bank of tests. There are many region-pairing rules (face versus neck, face versus background, left versus right cheek, eye versus eye, foreground object versus scene) and several estimator variants, such as different normal predictors and colour spaces.

Keyed selection. For an image $x$, the verifier draws $m$ tests $t_i = \mathrm{Bank}[F(K, h(x) \Vert i)]$, where $F$ is the puncturable function with key $K$ and $h$ is a perceptual hash. The image score is the mean inconsistency over the sampled tests:

$$s_K(x) = \frac{1}{m}\sum_{i=1}^{m} c\big(t_i(x)\big),$$

where $c$ is the scorer's inconsistency output. The perceptual hash ties the choice to image content, so a forger cannot pick an image that maps to easy tests without knowing $K$.

Delegation. The auditor punctures $K$ on a reserved index set $R$ and gives verifiers $K_R$. Verifiers reproduce every sampled test outside $R$ exactly. Tests inside $R$ remain unpredictable to them and to anyone who steals their key.

Baselines

The naive reference point is a standard full-image binary classifier, of the kind On Improving Cross-dataset Generalization of Deepfake Detectors uses as its base. Trained on FF++, such classifiers "typically struggle to exceed 0.60 AUC" on Celeb-DF according to that paper. We retrain one with the same backbone family and data so that we have an exact number of our own.

The strongest passive competitor is the RL-CNN from the same paper: an EfficientNet-v2-L backbone with a reinforcement-learning policy that picks test-time augmentations. Trained on FF++, it reports AUC 0.669 on Celeb-DF and 0.952 on DeeperForensics-1.0. We rerun it on our splits and frame selection with its reported configuration and the same hyperparameter-search budget as ours. Beating it means a Celeb-DF AUC above 0.669 from our scorer alone, or a statistically significant gain when our score is fused with its logit, at no loss on DeeperForensics.

The patch-truncated classifier of What Makes Fake Images Detectable? is the strongest "simpler generalization" competitor. No reported result was found on Lacuna for it on FF++→Celeb-DF, so we train it ourselves under the identical protocol.

For lighting-specific baselines we use the corneal test of Exposing GAN-Generated Faces (AUC 0.94 on FFHQ vs StyleGAN2) and the Disharmony network of Disharmony (AUC ≈0.70 on its iHarmony4 test set). We reproduce both on FFHQ/StyleGAN2 and iHarmony4 respectively before using them elsewhere.

Two ablations isolate the key ingredients. The first is a fixed public test set that uses the same bank and scorer but no key, so every test is known. The second replaces the puncturable function with two independent keys, one public and one reserved. Under the fixed public test set we expect clean AUC to match the keyed version and attacked AUC to collapse. Under the two-key version we expect detection identical to puncturing, and a difference only in key management.

Why not just…?

Why not just use a patch-based or augmentation-trained detector? Evidence in the file says these fail exactly where lighting matters. Detectors that read noise and local texture score 0.52–0.63 AUC on harmonized composites (Disharmony), and the best augmentation approach stops at 0.669 on Celeb-DF. This is still a real threat, because the patch classifier may already capture boundary lighting cues around the hair and skin. The head-to-head on Celeb-DF and the fusion experiment settle the question. If our score adds nothing on top of the patch model, the physics claim fails.

Why not just run fixed, public lighting checks? On clean benchmarks they may do as well as ours, and the fixed-test ablation measures exactly this. The case for keys rests entirely on the adaptive-attack experiment. If an attacker with white-box access to the fixed checker cannot drive its AUC down much more than ours, then the secrecy adds nothing and we will report that.

Why not just sign images at capture, or use a plain secret seed? Capture-time signatures and watermarks such as LIDMark need cooperation before release and say nothing about the large body of existing unsigned media, which is our target. A plain secret seed, or two independent keys, gives the same detection behaviour as puncturing; we expect this and test it as an ablation. Puncturing is justified only by key management. One key can be handed to many verifiers, each punctured on a different reserved set, and reserved tests can be revealed later without re-keying. If that operational benefit is not needed, the simpler version is the right choice, and the detection results carry over unchanged.

Proposed Experiments

Cross-dataset transfer. We train all learned components on FF++ and evaluate frame-level and video-level AUC on Celeb-DF and DeeperForensics-1.0, for our scorer alone and fused with the RL-CNN. We report results per FF++ manipulation type. We expect gains on identity swaps (Deepfakes, FaceSwap) and weakness on Face2Face and NeuralTextures, which re-render the target face under the target's own lighting. A null result there would be informative rather than fatal.

Reproduction outside faces. We reproduce the corneal result on FFHQ/StyleGAN2 and the Disharmony result on iHarmony4, then run our scorer on iHarmony4, RealHM and the Careaga composites. This checks that the bank works beyond faces and measures false positives on globally edited images.

Adaptive attack, the decisive check. An attacker fine-tunes a post-processing network, or runs gradient-based perturbations, against one of three targets: the public fixed checker, our checker given only the bank and the scorer weights, or our checker given the punctured key $K_R$. We measure AUC under attack together with the image quality (PSNR) the attacker must give up. The prediction that would falsify the idea is this: an attacker holding $K_R$ evades the reserved tests in $R$ as easily as the tests they can see. That outcome would mean evasion transfers across lighting tests, so secrecy buys nothing.

Cost of evasion. We test the remaining claim by measuring how much the attacker's image-quality budget must grow to evade all tests at once. If physics is the binding constraint, evading every test should cost markedly more quality than evading any single one.