Sparse Repellency for Shielded Generation in Text-to-image Diffusion Models

Michael Kirchhof, James Thornton, Pierre Ablin, Louis Béthune, Eugène Ndiaye, Marco Cuturi

Abstract

The increased adoption of diffusion models in text-to-image generation has triggered concerns on their reliability. Such models are now closely scrutinized under the lens of various metrics, notably calibration, fairness, or compute efficiency. We focus in this work on two issues that arise when deploying these models: a lack of diversity when prompting images, and a tendency to recreate images from the training set. To solve both problems, we propose a method that coaxes the sampled trajectories of pretrained diffusion models to land on images that fall outside of a reference set. We achieve this by adding repellency terms to the diffusion SDE throughout the generation trajectory, which are triggered whenever the path is expected to land too closely to an image in the shielded reference set. Our method is sparse in the sense that these repellency terms are zero and inactive most of the time, and even more so towards the end of the generation trajectory. Our method, named SPELL for sparse repellency, can be used either with a static reference set that contains protected images, or dynamically, by updating the set at each timestep with the expected images concurrently generated within a batch. We show that adding SPELL to popular diffusion models improves their diversity while impacting their FID only marginally, and performs comparatively better than other recent training-free diversity methods. We also demonstrate how SPELL can ensure a shielded generation away from a very large set of protected images by considering all 1.2M images from ImageNet as the protected set.

Introduction

Modern text-to-image diffusion models are incredibly powerful, but they suffer from two major reliability issues. First, they often exhibit a lack of diversity: even with different random seeds, models using Classifier-Free Guidance (CFG) frequently produce very similar images for the same prompt. Second, they are prone to memorization, where models "regurgitate" exact images from their training sets, raising significant copyright and privacy concerns.

Existing solutions often require expensive retraining or add heavy computational overhead during generation. This paper introduces SPELL (Sparse Repellency), a post-hoc, training-free method that is efficient and capable of scaling to protect against millions of images simultaneously.

The SPELL Method: Sparse Repellency

Instead of changing the model itself, SPELL modifies the "trajectory" of the image as it is being generated within the reverse SDE. The core innovation relies on a "shield" concept: the user defines a reference set of images to avoid. These could be images already generated in the current batch (to ensure diversity) or a massive database of protected training images (to prevent memorization).

At each step of the diffusion process, the model predicts the final image ($\hat{x}_0$). If this prediction falls within a predefined radius $r$ of a shielded image, SPELL applies a mathematical "nudge"—a repellency term—to push the trajectory outside the shield. Unlike other guidance methods that apply forces at every step, SPELL is sparse. It only activates when a "collision" with a shield is imminent, typically early in the generation process when the rough layout is decided. This leaves the later refinement stages untouched, preserving high image quality.

Qualitative comparison showing how SPELL increases visual diversity across different models and prompts.

Experimental Setup

The researchers evaluated SPELL across several high-profile models, including Stable Diffusion 3, MDTv2, and EDMv2, using the following datasets:

  • ImageNet-1k: Used for class-conditional generation and as a massive "protected set" of 1.2 million images to test anti-memorization capabilities.
  • CC12M (Conceptual 12M): A "soft-label" subset was created to measure how well a model captures the true diversity of a prompt.
  • MS COCO: Used for qualitative evaluation of complex text prompts.

To handle millions of protected images in real-time, the authors utilize Approximate Nearest Neighbor Search (Faiss) to quickly identify relevant shields during the generation path.

Key Results

SPELL demonstrates significant improvements in model reliability with minimal trade-offs:

  • Diversity Boost: SPELL consistently improved diversity metrics like Recall and Vendi Score. For example, on Latent Diffusion, Recall jumped from 0.236 to 0.289.
  • Anti-Memorization: When shielding the 1.2M ImageNet training images, SPELL reduced the rate of "near-duplicate" generations from 7.60% down to just 0.16%.
  • Quality Preservation: The FID (Fréchet Inception Distance) remained stable or improved in many cases, showing that increased diversity does not necessitate a loss in realism.
  • Efficiency: The computational overhead is negligible for standard batch sizes. Even when searching against the full 1.2M image set, generation time increased by only a few seconds per image on a single A100 GPU.

Insights and Limitations

A key insight of this work is treating diversity as a "collision" problem. By making trajectories self-avoiding and using the model's own expectation of the final result ($\hat{x}_0$) to guide the current step, the model effectively "sees into the future" to avoid unwanted outcomes.

However, some limitations remain. Currently, SPELL uses $L_2$ distance in the model's latent space, which may not always capture semantic similarity. Future work may explore repelling in more structured embedding spaces like CLIP or DINOv2. Additionally, while the current math is optimized for SDE-based samplers, extending these guarantees to deterministic ODE samplers is an open area for exploration.

Summary updated

Abstract

The increased adoption of diffusion models in text-to-image generation has triggered concerns on their reliability. Such models are now closely scrutinized under the lens of various metrics, notably calibration, fairness, or compute efficiency. We focus in this work on two issues that arise when deploying these models: a lack of diversity when prompting images, and a tendency to recreate images from the training set. To solve both problems, we propose a method that coaxes the sampled trajectories of pretrained diffusion models to land on images that fall outside of a reference set. We achieve this by adding repellency terms to the diffusion SDE throughout the generation trajectory, which are triggered whenever the path is expected to land too closely to an image in the shielded reference set. Our method is sparse in the sense that these repellency terms are zero and inactive most of the time, and even more so towards the end of the generation trajectory. Our method, named SPELL for sparse repellency, can be used either with a static reference set that contains protected images, or dynamically, by updating the set at each timestep with the expected images concurrently generated within a batch. We show that adding SPELL to popular diffusion models improves their diversity while impacting their FID only marginally, and performs comparatively better than other recent training-free diversity methods. We also demonstrate how SPELL can ensure a shielded generation away from a very large set of protected images by considering all 1.2M images from ImageNet as the protected set.
BibTeX
@misc{kirchhof2024sparse,
  author = {Michael Kirchhof and James Thornton and Pierre Ablin and Louis Béthune and Eugène Ndiaye and Marco Cuturi},
  title = {{Sparse Repellency for Shielded Generation in Text-to-image Diffusion Models}},
  year = {2024},
  eprint = {2410.06025},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2410.06025},
}
Figure