Abstract
The popularity of visual generative AI models like DALL-E 3, Stable Diffusion XL, Stable Video Diffusion, and Sora has been increasing. Through extensive evaluation, we discovered that the state-of-the-art visual generative models can generate content that bears a striking resemblance to characters protected by intellectual property rights held by major entertainment companies (such as Sony, Marvel, and Nintendo), which raises potential legal concerns. This happens when the input prompt contains the character's name or even just descriptive details about their characteristics. To mitigate such IP infringement problems, we also propose a defense method against it. In detail, we develop a revised generation paradigm that can identify potentially infringing generated content and prevent IP infringement by utilizing guidance techniques during the diffusion process. It has the capability to recognize generated content that may be infringing on intellectual property rights, and mitigate such infringement by employing guidance methods throughout the diffusion process without retrain or fine-tune the pretrained models. Experiments on well-known character IPs like Spider-Man, Iron Man, and Superman demonstrate the effectiveness of the proposed defense method.
Introduction
As visual generative AI matures into a multi-billion image-per-year industry, intellectual property (IP) holders like Sony, Marvel, and Nintendo have raised significant concerns regarding the unauthorized reproduction of their characters. While many AI platforms implement keyword filters to block explicit requests for characters like "Spider-Man," these safeguards are easily bypassed. Users can employ "lure prompts"—detailed descriptions of a character's appearance and traits—to induce the model into generating infringing content without ever mentioning a protected name.
This research addresses the systematic evaluation and mitigation of these "black-box" IP infringements. We investigate the pervasiveness of this "hidden" infringement and propose a method to suppress it while maintaining the model's creative utility.
Methodology: The TRIM Framework
We introduce TRIM (inTellectual pRoperty Infringement Mitigating), a defense framework designed to detect and steer generation away from protected IPs without requiring model retraining. The framework operates in three stages:
- Name-Based Filtering: An initial Large Language Model (LLM) filter catches explicit requests for protected character names.
- VLM-Based Detection: If a prompt passes the initial filter (e.g., a "lure prompt"), an image is generated. A Vision-Language Model (VLM), such as GPT-4V, then analyzes the output against a database of protected IPs to determine if the visual representation infringes on existing rights.
- Infringement Suppression via Guidance: If infringement is detected, the system triggers a second generation pass. It utilizes classifier-free guidance to "steer" the model. Specifically, the character's name is used as a "negative prompt," guiding the diffusion process away from the specific IP aesthetic while still adhering to the user's original descriptive text.
Experimental Setup
To evaluate the scope of the problem, we established a new benchmark focusing on six iconic IPs: Spider-Man, Iron Man, Incredible Hulk, Super Mario, Batman, and Superman. Our evaluation set consisted of 100 name-based prompts and 100 description-based "lure" prompts for each character. We tested seven state-of-the-art models, including various versions of Stable Diffusion, Kandinsky 2.1, DALL-E 3, and Midjourney.
Key Results and Insights
Our findings reveal that IP infringement is pervasive across leading models. In an undefended state, Midjourney produced infringing content 95% of the time when prompted with descriptions alone. DALL-E 3 (Microsoft Designer) showed an 84% infringement rate. This suggests that these models have deeply "memorized" protected IPs during their training phases.
The TRIM framework proved highly effective at mitigating these risks. On Stable Diffusion XL, Spider-Man infringement rates dropped from 76.6% to 5.8%. For Kandinsky 2.1, infringement for all five tested characters was reduced to 0.0%. Importantly, this mitigation did not significantly degrade image quality; CLIP scores, which measure prompt-image alignment, showed only a minor decrease, indicating that the resulting images remained relevant to the user's intent while becoming "legally safer."
Conclusion
The ease with which "lure prompts" can bypass current filters highlights a critical vulnerability in generative AI safety. Standard pixel-level metrics often fail to capture what a human instantly recognizes as a protected character. By using VLMs for detection and negative guidance for suppression, TRIM offers a scalable path toward responsible AI development that respects intellectual property without stifling creativity. Future work will focus on reducing computational overhead and extending these defenses to video generation models.
Summary updated