People increasingly add invisible protective noise to their photos before posting them. These perturbations, called unlearnable examples or cloaks, are tiny pixel changes designed to stop anyone from training a face-recognition or generation model on those images. Deepfake detectors are built on the same kind of scraped face data. As a result, some of the "real" faces a detector learns from will carry protective noise, even though nobody aimed that noise at the detector. Research on availability poisoning (making data useless for training) gives a reassuring rule of thumb called clean data leakage. If even a modest share of the training set is unprotected, the network has to learn the true features to fit those clean samples, and the protective shortcut stops mattering. We argue that this rule should fail for forgery detection specifically. A deepfake detector does not work from coarse semantics. It works from faint, high-frequency, local pixel residues, and that is the same signal channel protective noise lives in.
This proposal tests that prediction and explains it. We will build contaminated versions of the standard FaceForensics++ training set in which a controlled fraction of real faces carry protective perturbations of several families. We then measure two failure modes. The first is cross-dataset generalization: whether a detector trained on FaceForensics++ still works on other deepfake datasets. The second is false alarms on protected real photos, meaning the detector flags people who cloaked their photos as deepfakes. We also propose a measurable condition that predicts when leakage protects a model and when it does not: how much the perturbation overlaps with the detector's forgery-evidence direction. Finally, we test a cheap training-time fix and compare it against compression, diffusion purification and adversarial training.
Core Claim
Our claim is that clean data leakage neutralizes a partial shortcut only when the shortcut is roughly orthogonal to the features the task actually needs. Perturbation-Induced Linearization builds unlearnable data using only a linear surrogate classifier. Its gradient-orthogonality analysis shows that unlearnable perturbations do not suppress true features; they only offer an easier path. That is why clean samples dissolve their effect in object classification. In forgery detection, the useful features are local texture and upsampling irregularities. What Makes Fake Images Detectable? shows this by truncating backbones to small patches. The small "easier path" and the real path therefore occupy overlapping subspaces.
We predict that partially protected real data will do two things. It will teach detectors that a "structured high-frequency residual" is evidence, with a sign that depends on which class carried the noise more often. It will also shift decision thresholds in ways that clean leakage does not undo. Concretely, we expect protected real faces at test time to be over-flagged as fake, and we expect cross-dataset AUC to drop even at contamination rates of 5–20%, where classification-style leakage would predict near-full recovery.
To make "overlap" measurable, let $f(x)$ be the detector's fake-vs-real logit on image $x$, and let $\delta(x)$ be the protective perturbation applied to $x$. We define the alignment score
$$A = \mathbb{E}_{x}\left[\frac{|\langle \delta(x), \nabla_x f(x)\rangle|}{\|\delta(x)\|_2\,\|\nabla_x f(x)\|_2}\right],$$
where $\nabla_x f(x)$ is computed on a detector trained on clean data. The expectation runs over held-out real faces. $A$ therefore measures how much the perturbation points along the direction the clean detector already uses as forgery evidence. The claim is falsifiable: across combinations of perturbation family and detector architecture, $A$ should rank-correlate with the damage caused at a fixed contamination rate.
Relation to Prior Work
Two strands motivate the proposal. The availability-poisoning literature establishes both the strength and the fragility of protective noise. Unlearnable Examples introduced error-minimizing noise and showed it cuts protected-identity face-recognition accuracy on WebFace to 16%. APBench shows that such attacks can be removed by augmentation-based defenses. Separately, Provably Unlearnable Data Examples reports that even 1–5% clean leakage restores much of an attacker's performance. All of this work evaluates semantic classification, where leakage helps the learner. None of it asks what happens when the downstream task is itself a detector of low-level residues.
The forgery literature establishes that detectors are shortcut learners. CrossDF separates forgery-related, forgery-irrelevant and domain-specific information. GenD tunes only the normalization layers of a frozen foundation model and stresses that pairing each real video with its manipulated counterpart is what blocks shortcuts. Neither paper considers shortcuts that are injected unevenly into one class by third parties.
Two papers sit closest to our question. TSDF combines disrupting deepfake generation with poisoning the face detectors that generation pipelines rely on. It assumes the attacker retrains only on protected images, and it never examines passive detectors. AntifakePrompt tunes a small set of soft prompts in a vision-language model and checks robustness to deliberate data poisoning. It treats poisoning as an adversary aimed at the detector and does not vary the fraction of poisoned data.
Our setting differs from both. The noise is collateral rather than adversarial toward the detector, it is concentrated in the real class, and its rate is the independent variable.
One paper constrains our design. Can We Leave Deepfake Data Behind in Training Deepfake Detector? found that naively mixing two kinds of fakes can hurt, which it calls the 1+1<2 paradox, because the mixture disorganizes the detector's internal representation. This warns us that "just add more varied data" is not automatically a fix. It also suggests that partial contamination may be worse than full contamination, which we test directly.
Research Program
The first component is a contaminated-training benchmark built on FaceForensics++ at c23 compression. We apply protective perturbations to a fraction $\rho \in \{0, 5, 20, 50, 80, 100\}\%$ of real training frames, holding the fake frames fixed. The perturbation families are:
- sample-wise error-minimizing noise from Unlearnable Examples, optimized against an identity classifier (the realistic target of a cloak);
- PIL noise, which is cheap enough to generate at FaceForensics++ scale;
- IRP noise from Leveraging Imperfect Restoration, chosen because it is designed to survive compression and augmentation;
- TSDF-style protective noise.
We run a secondary condition in which fakes are re-synthesized from protected source frames on a subset of the manipulation methods whose code can be rerun. In that condition the perturbation partly survives into the fake class as well.
The second component is mechanistic. For each detector we compute the alignment score $A$. We also train linear probes on intermediate features to predict whether a perturbation is present, and we compare a truncated patch-level detector with a full-receptive-field one. The prediction is that the patch-level model, which relies most on local residues, suffers most. As a control, we run the same contamination protocol on an identity-classification task using the same faces. There, clean leakage should neutralize the noise. This separates "leakage fails in forensics" from "these perturbations are just unusually strong."
The third component is a fix. We call it perturbation-balanced pairing. Following GenD's insight that real/fake pairing prevents shortcuts, we apply perturbations drawn from the same families to fake frames at a matched rate. This makes the presence of protective noise statistically independent of the label, so it cannot serve as evidence in either direction. The key question is whether balancing with the actual cloak families matters, or whether generic noise at matched energy does just as well. If the answer is generic noise, the result is still useful, but the fix becomes an augmentation recipe rather than a contribution in its own right.
Baselines
The naive reference point is a detector trained on uncontaminated FaceForensics++ and evaluated on the same test suites. This defines the ceiling. We pair it with the "do nothing" detector trained on the contaminated pool without any mitigation, which defines the damage.
The strongest existing detector is GenD with LayerNorm-only tuning. It is the most relevant competitor because it reports the best average cross-dataset AUROC across 14 benchmarks. It also has the strongest a priori claim to immunity, since a frozen foundation encoder might ignore pixel-level noise. The file gives no per-dataset scalar for it; no reported Celeb-DF v2 number for GenD was found on Lacuna.
The strongest concrete number in the file comes from CrossDF, which reports 0.779 AUC on Celeb-DF v2 when trained on FaceForensics++ (CrossDF). We include CrossDF because its decomposition into domain and irrelevant streams might absorb the protective noise by design. We will reproduce both methods on our pipeline and compare against our reproduced clean numbers rather than the published ones. Each gets the same frames, face crops, training epochs and a matched hyperparameter search of equal trial count.
The mitigation baselines are:
- JPEG re-compression of all training frames;
- adversarial training;
- Sharpness-Aware Minimization, an optimizer that seeks flat loss minima, as used in Preserving Fairness Generalization in Deepfake Detection;
- diffusion purification of the kind APBench identifies as the most robust defense, applied to all training frames.
Each mitigation is applied on top of the same backbone. For fairness, each gets the same compute budget as our fix, plus one uncapped run to report its best case.
The ablation that isolates the key ingredient compares perturbation-balanced pairing with the true cloak families against balancing with Gaussian or uniform noise of matched $L_2$ energy. It also includes a held-out-family version, in which balancing uses three families and testing uses the fourth.
Our fix counts as beating the baselines only if it meets three conditions at $\rho = 20\%$:
- it recovers at least half of the gap between the contaminated and clean detectors in average cross-dataset AUC;
- it does so while holding the false-positive rate on protected reals within a few points of the clean detector;
- it does both at a small fraction of diffusion purification's cost.
Why not just…?
Why not just trust clean data leakage to fix it? This is the central alternative, and our experiments exist to test it. If detectors trained at $\rho = 20\%$ match clean detectors on cross-dataset AUC and on false positives for protected reals, our core claim is wrong, and we will report that as a clean negative result. The identity-classification control tells us whether leakage works on these exact faces and perturbations in the semantic setting. That lets us attribute any failure to the forensic task rather than the noise.
Why not just JPEG-compress everything? APBench reports that JPEG removes basic $L_\infty$ noise but not every family, and IRP specifically retained 4.14% accuracy against shortcut-squeezing defenses. More importantly, compression also erases the local residues that detectors need. We will measure this trade-off directly as cross-dataset AUC versus compression quality, with and without contamination. JPEG is a genuine threat to our proposal if a single quality setting restores most of the lost performance, and we will report it plainly if it does.
Why not just purify the training set with a diffusion model? APBench finds diffusion purification the most robust defense but also among the most expensive. Regenerating face frames with a generative model also plants generative fingerprints in the real class, which is exactly what a forgery detector must not learn to associate with "real." Our uncapped purification run measures both the recovery and any new false negatives on diffusion-made fakes.
Why not just use a frozen foundation model like GenD? Freezing the encoder limits what can be learned, but the LayerNorm parameters still adapt to whatever separates the classes in the training pairs. GenD also explicitly leaves adversarial perturbations unevaluated. If GenD shows little damage, that becomes a finding about foundation-model robustness, and our alignment score should explain it through low $A$.
Proposed Experiments
The primary evaluation trains on contaminated FaceForensics++ and tests on Celeb-DF v2, DFDC, DFD and DeeperForensics-1.0, reporting frame- and video-level AUC. The second test suite consists of protected real faces: Celeb-DF real frames plus CelebA and LFW faces, each cloaked with every perturbation family, including one family withheld from all training. On this suite we report the false-positive rate at a threshold fixed to 95% true-positive rate on clean data.
The primary output is a damage-versus-$\rho$ curve for each detector and perturbation family. It reveals whether damage is monotone in contamination or peaks at intermediate $\rho$, which would echo the 1+1<2 paradox.
There are three decisive checks.
The first is whether contamination at 5–20% causes a measurable drop in cross-dataset AUC or a rise in protected-real false positives while the identity-classification control recovers. This is the core test of whether leakage fails specifically in forensics.
The second is whether the alignment score $A$, computed before any contaminated training, rank-correlates with observed damage across all combinations of family and detector. This tests whether the proposed mechanism has predictive power beyond description.
The third is whether cloak-family balancing beats matched-energy generic noise, especially on the withheld family. If generic noise suffices, we will present the fix as a simple recommended augmentation and let the measurement and mechanism carry the contribution.
References
- Perturbation-Induced Linearization: Constructing Unlearnable Data with Solely Linear Classifiers
- What Makes Fake Images Detectable? Understanding Properties that Generalize
- Unlearnable Examples: Making Personal Data Unexploitable
- APBench: A Unified Availability Poisoning Attack and Defenses Benchmark
- Provably Unlearnable Data Examples
- Improving Cross-dataset Deepfake Detection with Deep Information Decomposition
- Deepfake Detection that Generalizes Across Benchmarks
- Boosting Active Defense Persistence: A Two-Stage Defense Framework Combining Interruption and Poisoning Against Deepfake
- AntifakePrompt: Prompt-Tuned Vision-Language Models are Fake Image Detectors
- Can We Leave Deepfake Data Behind in Training Deepfake Detector?
- Leveraging Imperfect Restoration for Data Availability Attack
- Preserving Fairness Generalization in Deepfake Detection