Misalignment Attack on Text-to-Image Models via Text Embedding Optimization and Inversion

Ziyong Du, Daizong Liu, Ping Zhou2025 · DOI

Abstract

Text embedding serves not only as a core component of modern NLP models but also plays a pivotal role in multimodal systems such as text-to-image (T2I) models, significantly facilitating user-friendly image generation through natural language instructions. However, while offering convenience, it also introduces additional risks. Misalignment issues in T2I models, whether caused by unintentional user inputs or targeted attacks, can negatively impact the reliability and ethics of these models. In this paper, we introduce TEOI, which fully considers the continuity and distribution characteristics of text embeddings. The framework directly optimizes the embeddings using gradient-based methods and then inverts them to obtain misaligned prompts of discrete tokens. The TEOI framework supports both text-modal and multimodal misalignment attacks, revealing the vulnerabilities of multimodal models that rely on text embeddings. Our work highlights the potential risks associated with embedding-based text representations in prevailing T2I models and provides a foundation for further research into robust and secure text-to-image generation systems.

Introduction

As text-to-image (T2I) models like Stable Diffusion and DALL-E become mainstream, ensuring they do not generate harmful, biased, or "Not-Safe-For-Work" (NSFW) content is a top priority. Most current defenses rely on prompt filtering—essentially a "banned words" list that blocks inputs containing offensive terms. However, these models do not process raw text directly; they process text embeddings (high-dimensional vectors).

Examples of misaligned prompts generating unexpected or harmful content while bypassing safety filters.

This paper addresses the "misalignment" problem: the possibility that a benign-looking text prompt can be mapped to a "harmful" embedding vector. If an attacker can find these "wolf in sheep's clothing" prompts, they can bypass filters entirely, rendering current safety mechanisms obsolete. We introduce TEOI, a framework that exposes this vulnerability by bridging the gap between what a prompt says and what the model "sees."

Methods

The TEOI framework utilizes a two-stage process to discover misaligned prompts:

  1. Text Embedding Optimization: Instead of heuristic searching, TEOI uses gradient-based optimization. Starting from a target image (e.g., an NSFW image or a specific face), the framework "nudges" the text embedding vector until the T2I model generates a visually similar output. To maintain readability, the authors use hybrid optimization, freezing a "semantic cue" and only optimizing a small portion of the prompt.
  2. Embedding Inversion: To convert these optimized vectors back into discrete, readable English tokens, TEOI employs a sophisticated inversion method based on the Vec2Text architecture.

A key innovation is the discovery that real text embeddings are sparse. By adding a distribution constraint loss (L1 norm) during optimization, TEOI ensures adversarial embeddings stay within the "natural" bounds of the model's latent space, making the attack significantly more effective.

Experimental Evaluation

The researchers evaluated TEOI across several standard and safety-focused benchmarks, including COCO for general tasks, the MMA (Multi-modal Adversarial) Dataset for NSFW content, and FFHQ for targeted identity manipulation. They also tested against extended NSFW categories using subsets of LAION and the Unsafe Diffusion dataset.

Key Results and Insights

TEOI achieved high success rates, with an average Attack Success Rate (ASR) of 77.2% on the COCO dataset and 66.4% on NSFW content, significantly outperforming previous methods like MMA-Diffusion.

A multimodal identity attack where TEOI manipulates both text and image inputs to change a subject's identity.

The framework also proved powerful in multimodal attacks, where input image pixels and text prompts are optimized simultaneously. Even "Safe Stable Diffusion," a model fine-tuned for robustness, remained vulnerable to TEOI-generated content. These results highlight that current safety filters are often too shallow, focusing on keywords while ignoring the "bad intent" hidden in the mathematical representation of the prompt.

Conclusion

The success of TEOI demonstrates that the alignment between human language and model embeddings is fragile. The study suggests that future defenses should move away from simple keyword filtering and toward semantic filtering, which checks if the meaning of a prompt (its embedding) is too close to known harmful concepts. While currently a white-box attack, TEOI provides a foundation for further research into robust and secure text-to-image generation systems.

Summary updated

Abstract

Text embedding serves not only as a core component of modern NLP models but also plays a pivotal role in multimodal systems such as text-to-image (T2I) models, significantly facilitating user-friendly image generation through natural language instructions.However, while offering convenience, it also introduces additional risks.Misalignment issues in T2I models, whether caused by unintentional user inputs or targeted attacks, can negatively impact the reliability and ethics of these models.In this paper, we introduce TEOI, which fully considers the continuity and distribution characteristics of text embeddings.The framework directly optimizes the embeddings using gradient-based methods and then inverts them to obtain misaligned prompts of discrete tokens.The TEOI framework supports both text-modal and multimodal misalignment attacks, revealing the vulnerabilities of multimodal models that rely on text embeddings.Our work highlights the potential risks associated with embedding-based text representations in prevailing T2I models and provides a foundation for further research into robust and secure text-to-image generation systems.
BibTeX
@misc{du2025misalignment,
  author = {Ziyong Du and Daizong Liu and Ping Zhou},
  title = {{Misalignment Attack on Text-to-Image Models via Text Embedding Optimization and Inversion}},
  year = {2025},
  doi = {10.18653/v1/2025.findings-emnlp.1200},
}
Figure