Abstract
Text-to-image generative models have been increasingly misused to create unauthorized malicious images of individuals. While existing defenses like Anti-DreamBooth add adversarial noise to protect images, these perturbations are often fragile and can be removed via adversarial purification methods like DiffPure. We propose High-Frequency Anti-DreamBooth (HF-ADB), a novel defense that concentrates strong adversarial perturbations in high-frequency image regions. By targeting edges and textures, HF-ADB achieves robustness against purification while maintaining visual quality. Experimental results demonstrate that HF-ADB effectively hinders malicious image generation even after adversarial cleaning, offering a more resilient solution for personal image protection.
Introduction
The rise of text-to-image models such as Stable Diffusion has introduced significant social challenges, particularly through "personalization" techniques like DreamBooth. These methods allow attackers to fine-tune models on a small set of unauthorized images to generate realistic deepfakes. To counter this, researchers have proposed adversarial defenses that "poison" images with subtle noise, causing fine-tuning to fail. However, these defenses are often vulnerable to adversarial purification—preprocessing steps like bilateral filtering or diffusion-based cleaning (e.g., DiffPure) that scrub low-magnitude noise.
Methodology: High-Frequency Targeting
High-Frequency Anti-DreamBooth (HF-ADB) addresses the fragility of previous methods by strategically distributing adversarial noise. Instead of a uniform noise budget, HF-ADB focuses intense perturbations on high-frequency areas, such as edges and complex textures, which are naturally more resilient to purification and less sensitive to human perception.
The HF-ADB pipeline involves three key steps:
- Edge Detection: A $3 \times 3$ Laplacian filter is applied to the input image to identify high-frequency components.
- Mask Generation: A binary mask is created by thresholding the edge values, typically isolating 3-5% of the image.
- Optimization via ASPL: The system employs Alternating Surrogate and Perturbation Learning (ASPL) to optimize noise against a surrogate model. A spatially varying noise budget is used: a baseline perturbation ($\eta$) for the whole image and a significantly stronger perturbation ($\eta_{mask}$) for the masked edge regions.
Experimental Setup
The effectiveness of HF-ADB was evaluated using the VGGFace2 dataset. The researchers sampled 400 images from 50 individuals, using four images per subject for protection and four as reference images for surrogate model training. All images were processed at a $512 \times 512$ resolution.
Results and Discussion
Qualitative analysis reveals that HF-ADB is significantly more robust than standard Anti-DreamBooth. When subjected to purification via bilateral filters, the baseline defense is often neutralized, allowing the generation of clean, realistic images. In contrast, images protected by HF-ADB continue to produce "unnatural patterns" and severe artifacts in the generated output, even after purification attempts.
A key insight from the study is the limitation of standard computer vision metrics. While visual results clearly showed the success of the defense, automated metrics like Face Detection Success Ratio often failed to capture the qualitative degradation of the generated images. This highlights a need for more specialized evaluation frameworks for adversarial defenses in generative AI.
Limitations and Future Work
While effective for human faces, HF-ADB showed reduced efficacy when applied to artworks (e.g., paintings by Van Gogh). The inherent texture and color complexity of artistic styles may naturally absorb or mask the adversarial noise, making it less disruptive to the fine-tuning process. Additionally, while the noise is concentrated on edges, it may still be perceptible under close scrutiny. Future research will focus on improving the imperceptibility of the noise and extending the defense's robustness across diverse image domains.
Summary updated