Abstract
Large-scale generative models often inadvertently memorize copyrighted content from their training data, posing significant legal and ethical challenges. We present UniForget, a novel "blind" de-memorization technique that mitigates the generation of sensitive content without requiring prior knowledge of specific target concepts. By pruning a small subset of model parameters using only neutral prompts, UniForget successfully decouples memorized "spikes" from general generative capabilities. Our experiments on the FLUX model demonstrate the effective suppression of over 2,000 copyright-sensitive concepts while maintaining high image quality and general-purpose utility.
Introduction
State-of-the-art generative models like Stable Diffusion 3 and FLUX are trained on massive web-scale datasets. As these models scale, they increasingly tend to "memorize" training samples, reproducing copyrighted characters, celebrity likenesses, and proprietary artwork nearly verbatim. Existing solutions typically rely on targeted unlearning—which requires identifying every concept to be forgotten—or guidance manipulation, which adds computational overhead and often degrades output quality.
UniForget addresses these limitations by enabling a model to "unconsciously forget" memorized content. It operates under the premise that memorization is a distinct mathematical phenomenon that can be surgically removed without ever exposing the model to the copyrighted data during the mitigation process.
The UniForget Framework
Theoretical Foundations
UniForget is built upon two primary hypotheses:
- The Spike Hypothesis: Memorized content exists as high-probability "spikes" in the model's embedding space. These spikes are mathematically distinct and have higher magnitudes compared to the "neutral subspace" used for general generation.
- The Selective Parameter Hypothesis: A small, specific subset of the model's weights (primarily in Feed-Forward Networks and Normalization layers) is responsible for these memorization spikes.
Implementation via Model Pruning
Instead of traditional fine-tuning, UniForget utilizes Model Pruning. The researchers train a learnable mask ($M$) to identify and deactivate "memorization neurons." The training process is remarkably simple and "blind," requiring only 256 neutral, safe prompts (e.g., "a photo of a park") sampled from LAION-2B.
The pruning is guided by a dual-objective loss function that balances reconstruction (ensuring the model still generates neutral images effectively) and sparsity (minimizing active parameters). Because memorization spikes are parameter-hungry and distinct from general features, they are the first elements to be pruned away when the model is forced to maintain quality with fewer active weights.
Experimental Results
Qualitative Analysis
Testing on the FLUX model reveals that UniForget effectively transforms specific copyrighted characters into generic, high-quality versions. As pruning strength increases from "Weak" to "Strong," the model progressively loses the ability to render specific likenesses while retaining the core composition and artistic quality of the prompt.
Quantitative Performance
The method successfully suppressed over 2,000 different copyright-sensitive concepts. On the Anime dataset, detection accuracy of copyrighted features dropped significantly from 77.6% in the original model to 27.4% under strong pruning. Crucially, general generative performance remained stable; the FID score on the COCO dataset only shifted from 32.0 to 38.9 in the strongest setting, and could be further optimized to 31.5 with a brief retraining phase that did not re-introduce the memorized content.
Conclusion
UniForget represents a significant step forward in responsible AI development. By treating memorization as a "bug" that can be surgically pruned, it offers a scalable, efficient, and blind method for protecting intellectual property in generative models. Future work will focus on extending this architecture-agnostic approach to other models like Stable Diffusion 3 and Visual Autoregressive (VAR) models to further prove its universality.
Summary updated