Evaluating and Mitigating IP Infringement in Visual Generative AI

Zhenting Wang, Chen Chen, Vikash Sehwag, Minzhou Pan, Lingjuan Lyu

TL;DR

Visual generative AI models like DALL-E 3 and Midjourney can be easily "lured" into infringing on intellectual property (IP) by using descriptive prompts that avoid character names. To combat this, the authors propose TRIM (inTellectual pRoperty Infringement Mitigating), a defense framework that uses Vision-Language Models to detect infringing outputs and employs classifier-free guidance to "steer" the generation process away from protected characters without requiring model retraining.

What Problem Does This Paper Address?

As visual AI becomes a multi-billion image-per-year industry, major entertainment companies (Sony, Marvel, Nintendo) are increasingly concerned about their characters being reproduced without permission. While many AI providers block specific keywords (e.g., "Spider-Man"), users can bypass these filters using "lure prompts"—detailed descriptions of a character’s appearance and traits.

The authors identify a critical gap: there is no systematic way to evaluate or mitigate these "black-box" IP infringements where the user doesn't have access to the model's internal weights. The paper seeks to answer: How pervasive is this "hidden" infringement, and can we stop it without ruining the model's creative utility?

Methods

The researchers introduced a two-part approach: evaluation via "lure prompts" and mitigation via the TRIM framework.

  1. Lure Prompt Generation: They used GPT-4 to create highly descriptive prompts that characterize a hero (e.g., "a superhero in a sleek, skin-tight suit, primarily red with distinctive blue patterns...") without ever mentioning the name.
  2. TRIM (The Defense):
    • Name Blocking: A standard LLM filter to catch explicit name-based requests.
  3. Output Infringement Detection: If a prompt passes the first filter, an image is generated. A Vision-Language Model (GPT-4V) then analyzes the image against a database of protected IPs to see if it "looks like" a character it shouldn't.
  4. Infringement Suppression: If infringement is detected, the system triggers a second generation pass. It uses classifier-free guidance, a technique that usually guides a model toward a prompt. TRIM uses it to guide the model away from the detected character (using the character's name as a "negative prompt") while still following the user's original descriptive text.

Datasets

The authors established a new benchmark for IP infringement evaluation:

  • Target Characters: Six iconic, high-grossing IPs: Spider-Man, Iron Man, Incredible Hulk, Super Mario, Batman, and Superman.
  • Evaluation Set: 100 name-based prompts and 100 description-based "lure" prompts for each character.
  • Models Tested: Seven state-of-the-art models, including Stable Diffusion (v1.5, XL, XL-turbo), Kandinsky 2.1, DALL-E 3 (API, ChatGPT, and Microsoft Designer versions), and Midjourney.

Key Insights and Ideas

  • The "Lure" is Highly Effective: Even without names, models like Midjourney and DALL-E 3 produced infringing content nearly 100% of the time for certain characters. This suggests the models have "memorized" these IPs deeply during training.
  • Algorithmic Metrics vs. Human Perception: The authors found that standard metrics like L2 distance or LPIPS (which measure pixel-level similarity) are poor indicators of IP infringement. An image can be mathematically "distant" from a reference photo but still be instantly recognizable as Mickey Mouse to a human.
  • Guidance as a Scalpel: The most novel part of TRIM is using the character name as a negative condition during diffusion. This allows the model to fulfill the user's request for a "superhero in a red suit" while specifically avoiding the "Spider-Man" aesthetic.

Key Results

  • Pervasive Infringement: In the "Undefended" state, Midjourney had an average infringement rate of 95% across all characters using only descriptions. DALL-E 3 (Microsoft Designer) followed closely at 84%.
  • TRIM Effectiveness: The defense drastically reduced infringement. For example, on Stable Diffusion XL, Spider-Man infringement dropped from 76.6% to 5.8%. On Kandinsky 2.1, infringement for all five tested characters dropped to 0.0%.
  • Preserving Quality: The authors used CLIP scores to measure how well the final image matched the user's prompt. TRIM only caused a slight dip in CLIP scores (e.g., from 34.17 to 30.14 for Spider-Man), meaning the images remained high-quality and relevant, just "legally safer."

Limitations and Future Work

  • VLM Dependency: The defense is only as good as the Vision-Language Model used for detection. If GPT-4V fails to recognize a character, the infringement will pass through.
  • Computational Overhead: Because TRIM requires a detection step and potentially a second generation pass, it increases the time and cost per image.
  • Scalability: The current study focused on six major characters. Scaling this to thousands of protected IPs would require a more automated way to manage the "protected list" and negative prompts.
  • Video Infringement: While the paper mentions Stable Video Diffusion, the primary focus and defense are optimized for static images. Extending TRIM to temporal consistency in video is a suggested next step.

Summary updated

Abstract

The popularity of visual generative AI models like DALL-E 3, Stable Diffusion XL, Stable Video Diffusion, and Sora has been increasing. Through extensive evaluation, we discovered that the state-of-the-art visual generative models can generate content that bears a striking resemblance to characters protected by intellectual property rights held by major entertainment companies (such as Sony, Marvel, and Nintendo), which raises potential legal concerns. This happens when the input prompt contains the character's name or even just descriptive details about their characteristics. To mitigate such IP infringement problems, we also propose a defense method against it. In detail, we develop a revised generation paradigm that can identify potentially infringing generated content and prevent IP infringement by utilizing guidance techniques during the diffusion process. It has the capability to recognize generated content that may be infringing on intellectual property rights, and mitigate such infringement by employing guidance methods throughout the diffusion process without retrain or fine-tune the pretrained models. Experiments on well-known character IPs like Spider-Man, Iron Man, and Superman demonstrate the effectiveness of the proposed defense method. Our data and code can be found at https://github.com/ZhentingWang/GAI_IP_Infringement.
BibTeX
@misc{wang2024evaluating,
  author = {Zhenting Wang and Chen Chen and Vikash Sehwag and Minzhou Pan and Lingjuan Lyu},
  title = {{Evaluating and Mitigating IP Infringement in Visual Generative AI}},
  year = {2024},
  eprint = {2406.04662},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2406.04662},
}

Loading related content...

Figure