Sparse Repellency for Shielded Generation in Text-to-Image Diffusion Models

Michael Kirchhof, James Thornton, Pierre Ablin, Louis Béthune, Eugene Ndiaye, marco cuturiSubmission to ICLR 2025 · Rejected

TL;DR

Diffusion models frequently suffer from two operational flaws: they produce repetitive, near-identical outputs when prompted repeatedly, and they risk regurgitating exact training samples. To fix this without retraining, SPELL (SPArse rEpelLency) introduces a geometrically grounded, training-free guidance term that steers diffusion trajectories away from "shielded" reference regions. By sparsely nudging predicted endpoints only when they drift too close to protected or concurrently generated images, SPELL boosts generation diversity and prevents training data replication with minimal precision or compute overhead.

What Problem Does This Paper Address?

Despite the success of modern text-to-image and class-conditioned diffusion models, deploying them reveals two critical challenges:

  1. Lack of Diversity: Under standard sampling—especially when boosted by classifier-free guidance (CFG)—models often settle into narrow modes, generating only minor variations of the same composition across repeated runs with identical prompts.
  2. Data Memorization / Replication: Diffusion models can memorize and reproduce training samples, raising copyright, privacy, and licensing concerns.

Prior diversity solutions (such as continuous particle interactions or condition annealing) frequently disrupt the sampling trajectory throughout the entire process, harming sample fidelity or drastically slowing inference. The authors tackle how to enforce sample diversity and guarantee avoidance of specific reference sets post-hoc, efficiently, and without degrading image quality.

Methods

SPELL modifies the reverse stochastic differential equation (SDE) during generation using a simple geometric correction:

  • Endpoint Prediction via Tweedie’s Formula: At any reverse step $t$, the denoiser predicts the final clean image $\hat{x}_0 = D_\theta(t, x_t) \approx \mathbb{E}[X_0 \mid X_t = x_t]$.
  • Hard-Sphere Repellency: The target is protected by defining exclusion "shields" of radius $r$ around reference points $z_k$: $$\mathcal{S} = \bigcup_k \{x : \|x - z_k\|_2 \le r\}$$
  • Sparse Correction ($\Delta$): If the trajectory is predicted to land within a shield ($\|\hat{x}_0 - z_k\|_2 < r$), SPELL adds a radial repulsive term to push $\hat{x}_0$ back onto the boundary of the ball: $$\Delta = \sum_{k=1}^K \text{ReLU}\left(\frac{r}{\|\hat{x}_0 - z_k\|_2} - 1\right)(\hat{x}_0 - z_k)$$ An optional overcompensation multiplier $\lambda \ge 1$ (e.g., $\lambda = 1.6$) helps push trajectories cleanly into adjacent attraction basins earlier in the sampling process.

  • Flexible Modes:

  • Intra-batch / Inter-batch Diversity: The reference points $z_k$ are dynamically populated by other images being generated in the same batch or across previous batches for the same prompt.
  • Static Image Shielding: The set $\{z_k\}$ contains protected real images (e.g., copyrighted or training data). When scaling to large reference sets, SPELL couples with fast approximate nearest-neighbor search (via FAISS IndexIVFFlat).

Datasets

  • Soft-Label CC12M: A newly curated subset of Conceptual 12M comprising 5,554 distinct captions with 4 to 128 corresponding ground-truth images each (554 validation, 5,000 test captions; 41,596 total images), constructed to evaluate one-to-many diversity while filtering out generic titles, web-scraping artifacts, and identifiable individuals.
  • ImageNet-1k: Used both for class-conditional generation benchmarks and as a massive static shielded set (all 1.2M training images) to test replication prevention.
  • MS COCO: Used for zero-shot qualitative text-to-image evaluations on un-trained prompts.

Key Insights and Ideas

  • Intervene on Future Expectations, Not Current States: Unlike Particle Guidance (which continuously repels noisy latent representations $x_t$), SPELL projects trajectories based on the expected denoised output $\hat{x}_0$.
  • Sparsity by Design: Because the ReLU activation zeroes out whenever $\|\hat{x}_0 - z_k\|_2 \ge r$, SPELL is completely inactive across most steps. Trajectories headed toward distinct modes are left untouched.
  • Early Action Prevents Late Artifacts: Denoising dynamics outline coarse global structures early ($t \in [0.6, 1.0]$) and refine local texture late ($t < 0.5$). SPELL triggers primarily in early timesteps (declining to only ~20% active runs by $t=0.6$), allowing the native diffusion score to refine high-frequency details later without interference or visual artifacts.
  • Doob’s $h$-Transform Connection: The authors mathematically link SPELL to conditioning via Doob’s $h$-transform, showing it can be seen as an exact geometric projection alternative to softer, hard-to-tune Gaussian diffusion posterior sampling (DPS).

Key Results

  • Diversity vs. Quality Pareto Fronts: Across Stable Diffusion 3, Latent Diffusion, EDMv2, MDTv2, and Simple Diffusion, SPELL consistently boosts diversity metrics (Recall, Vendi Score) with minimal impact on Precision and FID. On Latent Diffusion (CC12M), SPELL delivers better Pareto trade-offs across Recall/Precision, Coverage/Density, and Vendi/CLIP score than competing methods like CADS, Interval Guidance (IG), and Particle Guidance (PG).
  • Training Data Shielding at Scale: When shielding against all 1.2 million ImageNet-1k training samples:
  • Baseline EDMv2 generated images within an $L_2$ radius $r=60$ of a training image 7.60% of the time.
  • Adding SPELL reduced this replication rate to 0.16% without retraining or having to generate-and-discard candidates.
  • Minimal Computational Overhead: In intra-batch diversity settings ($K \le 128$), generation time remains virtually indistinguishable from the baseline (e.g., 2.94s vs. 2.93s per image on Simple Diffusion).

Limitations and Future Work

  • Shield Overlaps: SPELL’s strict guarantee of landing outside shielded balls mathematically assumes disjoint regions. If shields overlap heavily, exact projections would require solving a quadratic program.
  • Metric Space Sensitivity: Currently, distances are computed in RGB space or VAE latent space ($L_2$ norm), capturing low-level visual and color composition. Exploring semantic embeddings (such as DINOv2 feature space) could steer trajectories toward higher-level semantic diversity.
  • Scalability Bottlenecks: When repelling from millions of static images, approximate nearest-neighbor retrieval introduces inference latency (e.g., increasing generation time from 2.4s to 4.6–13.5s per image depending on Voronoi probe counts).

Summary updated

Abstract

The increased adoption of diffusion models in text-to-image generation has triggered concerns on their reliability. Such models are now closely scrutinized under the lens of various metrics, notably calibration, fairness, or compute efficiency. We focus in this work on two issues that arise when deploying these models: a lack of diversity when prompting images, and a tendency to recreate images from the training set. To solve both problems, we propose a method that coaxes the sampled trajectories of pretrained diffusion models to land on images that fall outside of a reference set. We achieve this by adding a simple repellency term to the diffusion SDE throughout the generation trajectory, that is triggered whenever it is expected to land too closely to an image in the shielded reference set. Our method is sparse in the sense that these repellency terms are mostly zero and inactive, even more so towards the end of the generation trajectory. Our method, named SPELL for sparse repellency, can be used either with a static reference set that contains protected images, or dynamically, by updating the reference set at each timestep with the expected images concurrently generated within a batch. We show that adding SPELL to popular diffusion models improves their diversity while impacting their FID only marginally, and performs comparatively better than other recent training-free diversity methods. Moreover, we demonstrate how SPELL can ensure a shielded generation away from a very large set of protected images by considering all 1.2M images from ImageNet as the protected set.
BibTeX
@misc{kirchhof2024sparse,
  author = {Michael Kirchhof and James Thornton and Pierre Ablin and Louis Béthune and Eugene Ndiaye and marco cuturi},
  title = {{Sparse Repellency for Shielded Generation in Text-to-Image Diffusion Models}},
  year = {2024},
  eprint = {2410.06025},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2410.06025},
}

Loading related content...

Figure