Artists and ordinary people now add invisible noise to their photos so that image generators cannot copy their style or face. Tools such as Glaze, Mist and MetaCloak work this way, and so does AntiPure. A second wave of work removes that noise. These methods are called purifiers: tools that clean an image by pushing it back toward what natural images look like, usually with a diffusion model (a generator that learns by gradually turning random noise into images). The strongest purifiers do not simply blur the noise away. They find it first. IMPRESS and DIRP both notice that protected images reconstruct badly when passed through a diffusion model's encoder or through DDIM inversion (running the generator's denoising process backwards to recover the noise that would produce the image). They then optimize the image until that reconstruction error goes away. The protection's own side effect guides the purifier to it.
This note proposes to test one question directly. Can a protective perturbation be crafted to leave no reconstruction signature and still disrupt personalization? The alternative is that the two goals are fundamentally in conflict. We would find out by building a protection that suppresses its own signature while being optimized across a family of purifiers. We would then judge it with a protocol the field lacks. That protocol withholds some purifiers from the defender during crafting. It reports full curves of disruption against defender budget and purifier effort. It also measures how much damage the purifier must do to clean images to succeed. Either answer is useful. A positive answer gives protections a principled design target. A negative one is a clear impossibility result that users of these tools deserve to know.
Core Claim
Existing protections fail against adaptive purifiers for a specific reason: they sit off the natural-image manifold, and that is detectable. Making noise robust through expectation over transformation (EOT) does not fix this. EOT means optimizing the noise to work on average across random transformations of the image, and MetaCloak and LowKey use it over blur, crop and JPEG. It makes noise survive those transformations, but it does not make the noise invisible to a reconstruction-error detector.
We claim that the right unit of evaluation is the residual protection after a purifier that is allowed to see the protection method. We also claim that the right design target is a perturbation whose reconstruction error matches that of clean images. We will test two hypotheses. First, EOT over a distribution of diffusion purifiers, combined with a penalty on the reconstruction gap, keeps measurably more protection against purifiers held out from crafting than either ingredient alone. Second, whatever protection remains forces the purifier into a regime where it degrades clean images almost as much as it restores protected ones.
Concretely, the defender solves
$$\max_{\|\delta\|_\infty \le \epsilon} \; \mathbb{E}_{P \sim \mathcal{P}_{\text{train}}}\big[ D(P(x+\delta)) \big] \;-\; \lambda \, \big[ R(x+\delta) - R(x) \big]_+ .$$
Here $x$ is the image to protect and $\delta$ is the added noise with pixel budget $\epsilon$. $\mathcal{P}_{\text{train}}$ is a distribution over purifiers: JPEG, blur, stochastic diffuse-then-denoise at random noise levels, and short DDIM inversion-reconstruction. $D$ is the disruption loss on personalization, the maximized denoising error used by MetaCloak. $R$ is the DDIM reconstruction error that DIRP exploits, and $[\cdot]_+$ keeps only the positive part. The weight $\lambda$ trades disruption against stealth toward the purifier.
Research Program
The work has three parts.
The first part is an evaluation harness for protective perturbations under adaptive purification. It extends the protocol of Evaluating Adversarial Protections for Diffusion Personalization, which compares eight protections under plain DreamBooth fine-tuning with no purification. We add a fixed battery of purifiers to that protocol, applied before personalization. We borrow the principle of On Evaluating Adversarial Robustness and of Benchmarking Adversarial Robustness on Image Classification that single-budget numbers mislead. Every result is therefore a surface: disruption plotted against defender budget $\epsilon$ from 4/255 to 16/255, and against purifier effort. Purifier effort means optimization steps for IMPRESS and DIRP, and diffusion noise level for stochastic purification.
The harness also reports a quantity we call purifier cost. It is the drop in quality metrics (Fréchet Inception Distance and LPIPS perceptual similarity) when the same purifier is run on unprotected images. A protection that can only be removed by a purifier that also ruins clean images has succeeded in a meaningful sense. That outcome is what AntiPure hints at when it reports protection growing with more purification iterations.
The second part is the signature-suppressed EOT protection above. Backpropagating through DDIM inversion is expensive and prone to noisy or vanishing gradients. We will therefore use short inversion chains as the in-loop surrogate. We will reuse the gradient-correctness practices of Unlocking The Potential of Adaptive Attacks on Diffusion-Based Purification, namely time-consistent gradients and reusing the same noise samples across forward and backward passes. That paper also supplies a lesson we must respect: randomness alone only hides gradients, so the defense cannot count on stochastic purification to protect it.
The third part is a held-out purifier protocol. The defender crafts against $\mathcal{P}_{\text{train}}$, and evaluation uses purifiers outside it. In the main split, DIRP and IMPRESS are both held out, and the protection sees only generic diffuse-and-denoise and short inversion. This is the step that separates real robustness from overfitting to the attacker the defender happened to model. That kind of overfitting is exactly the surrogate problem that transferability benchmarks document for classifiers, in Devling into Adversarial Transferability on Image Classification.
Finally, we close the loop with a second-round adaptive purifier. It knows the new objective and tries alternative signatures, such as latent-space statistics or high-frequency patch energy. It also tries brute force, such as heavy diffusion noise. This round follows the demand in the adaptive-attack literature that defenses be evaluated against attackers who know them.
Baselines
The naive reference point is an unprotected image under each purifier and personalization pipeline. It sets the ceiling on mimicry success. IMPRESS reports style mimicry of roughly 90% without protection on WikiArt. An unpurified protected image sets the floor for each protection.
The strongest existing protection against purification is AntiPure. It crafts noise that attacks the diffusion purifier itself through patch-wise frequency guidance and erroneous timestep guidance. It is the closest competitor because it is the only method in the file designed to make purification backfire. However, it was not tested against optimization-based purifiers such as DIRP. For AntiPure's metric values, no reported result found on Lacuna. We will run it from the paper's recipe on its own 50-identity, 12-image protocol.
MetaCloak is the strongest EOT-based protection. It uses meta-learning over surrogate models plus EOT over flip, crop and blur. MetaCloak reports lowering Subject Detection Score from 0.903 to 0.432 on VGGFace2. We also include Glaze-style and Mist-style low-level protections, which DIRP reports breaking.
On the purifier side, the strongest adaptive attackers are these:
- DIRP, from the DIRP codebase. It reduces DIAGNOSIS watermark accuracy from about 100% to below 2%.
- IMPRESS. It restores style mimicry on Glaze-protected WikiArt from about 42% to 87%, and to about 80% against an adaptive Glaze.
Fairness works as follows. Every protection gets the same $\epsilon$ grid, the same Stable Diffusion surrogate and the same crafting compute in GPU-hours. MetaCloak's EOT loop is allowed the same number of transformation samples per step as ours. Every purifier gets the same step budget sweep. DIRP's default 500 steps is the top of that sweep.
Two ablations isolate the key ingredients. One removes the reconstruction-gap penalty, setting $\lambda=0$. This gives plain EOT over purifiers, which is the obvious "EOT over diffusion" idea. The other keeps the penalty but uses MetaCloak's classical transformation set in place of purifiers.
Beating the baselines means one of two things on held-out DIRP and IMPRESS. The first is a larger area under the disruption-versus-purifier-effort curve at matched stealth, measured by LPIPS and the colour-difference metric CIEDE2000, on at least two of VGGFace2, CelebA-HQ and WikiArt. The second is that recovering the unprotected mimicry level requires a purifier whose purifier cost exceeds a pre-set threshold.
Why not just…?
Why not just benchmark existing protections against IMPRESS and DIRP, without a new method? That benchmark is part one of this proposal and is valuable by itself. It would not answer the design question, though. Without a candidate that removes the reconstruction signature, a uniformly bad table cannot tell us whether the failure is fundamental or an artefact of how current protections are built. The harness is necessary, and the method is the probe.
Why not just add stronger augmentations to EOT, as MetaCloak and LowKey do? IMPRESS reports that JPEG, noise and resizing are "not enough" to break Glaze and PhotoGuard, while model-aware purification breaks them. The mirror image is that surviving those augmentations says nothing about surviving model-aware purification. Our classical-EOT-plus-penalty ablation tests this directly.
Why not just raise $\epsilon$? Ye et al. show stealth metrics degrade sharply from 4/255 to 16/255. DIRP argues that larger perturbations push images further off-manifold and make the reconstruction explosion easier to find. Our full $\epsilon$ curves will show whether higher budgets buy residual protection or only visible artefacts.
Why not just do EOT over diffusion purifiers, without the gap penalty? This is the real threat. It may be enough, and AntiPure's growing effectiveness with more purification suggests purifier-aware crafting has teeth. The $\lambda=0$ ablation on held-out purifiers settles it. If plain purifier-EOT matches the full method there, we drop the signature claim and report purifier-EOT with the held-out protocol as the finding.
Decisive Checks
The first check is a signature audit. For each protection, we plot the distribution of DDIM reconstruction error on protected versus clean images. For our method, a simple threshold detector should approach chance, measured by area under the ROC curve. If it does not, the penalty failed and the downstream results cannot be attributed to stealth.
The second check is the held-out curves. We plot disruption against DIRP and IMPRESS steps on WikiArt style mimicry and on VGGFace2 identity. Identity is measured by Subject Detection Score and by the Calibrated Identity Ratio from the Proactive Defense Benchmark. Generation quality is measured with FID, BRISQUE, LIQE and CLIP-IQAC.
The third check is the second-round adaptive purifier together with purifier cost. If a purifier that knows about the penalty restores mimicry at low cost to clean images, the result is negative. That would be strong evidence that imperceptible protection cannot survive adaptive purification, and we would report it as such.
Compute is the main practical risk. IMPRESS takes about 141 seconds per image and DIRP uses 500 steps. We therefore run the full sweep on the 50-identity subsets and a WikiArt subset, and reserve the largest budgets for a representative sample.
References
- IMPRESS: Evaluating the Resilience of Imperceptible Perturbations Against Unauthorized Data Usage in Diffusion-Based Generative AI
- DDIM Inversion as a Perturbation Amplifier: Breaking Mimicry Protection via Reconstruction Error Minimization
- Evaluating Adversarial Protections for Diffusion Personalization: A Comprehensive Study
- On Evaluating Adversarial Robustness
- Benchmarking Adversarial Robustness on Image Classification
- Towards Robust Defense against Customization via Protective Perturbation Resistant to Diffusion-based Purification
- DiffBreak: Is Diffusion-Based Purification Robust?
- Devling into Adversarial Transferability on Image Classification: Review, Benchmark, and Evaluation
- MetaCloak: Preventing Unauthorized Subject-Driven Text-to-Image Diffusion-Based Synthesis via Meta-Learning
- Proactive Defense Benchmark against Deepfake Generation