NaviDet: Efficient Input-level Backdoor Detection on Text-to-Image Synthesis via Neuron Activation Variation

Shengfang Zhai, Jiajun Li, Yue Liu, Huanran Chen, Zhihua Tian, Wenjie Qu, Qingni Shen, Ruoxi Jia, Yinpeng Dong, Jiaheng Zhang

Abstract

In recent years, text-to-image (T2I) diffusion models have garnered significant attention for their ability to generate high-quality images reflecting text prompts. However, their growing popularity has also led to the emergence of backdoor threats, posing substantial risks. Currently, effective defense strategies against such threats are lacking due to the diversity of backdoor targets in T2I synthesis. In this paper, we propose NaviDet, the first general input-level backdoor detection framework for identifying backdoor inputs across various backdoor targets. Our approach is based on the new observation that trigger tokens tend to induce significant neuron activation variation in the early stage of the diffusion generation process, a phenomenon we term Early-step Activation Variation. Leveraging this insight, NaviDet detects malicious samples by analyzing neuron activation variations caused by input tokens. Through extensive experiments, we demonstrate the effectiveness and efficiency of our method against various T2I backdoor attacks, surpassing existing baselines with significantly lower computational overhead. Furthermore, we rigorously demonstrate that our method remains effective against potential adaptive attacks.

Introduction

Text-to-Image (T2I) diffusion models, such as Stable Diffusion, have revolutionized content creation but introduced critical security vulnerabilities. Attackers can embed backdoors into these models, where a specific "trigger" token in a prompt causes the model to generate unauthorized or harmful content. Defending against these attacks is challenging because triggers often only modify subtle details (e.g., changing a cat to a zebra) rather than dominating the entire output. Furthermore, existing defenses are computationally prohibitive, often requiring multiple full image generation cycles to detect a single malicious prompt.

Methodology: The NaviT2I Framework

NaviT2I is an input-level defense mechanism designed to act as a real-time firewall. It is built upon the discovery of Early-step Activation Variation: backdoor trigger tokens cause massive, unique spikes in internal neuron activity during the very first steps of the denoising process. Theoretically, the text prompt's influence is strongest at the start of sampling; as the process continues, the evolving image data "washes out" the trigger's signal.

The NaviT2I pipeline: from token-level augmentation to single-step activation analysis and outlier detection.

The NaviT2I pipeline involves four key stages:

  1. Token Masking: Sequentially masking non-stopwords in the input prompt to isolate individual token contributions.
  2. Single-Step Analysis: Running only the first denoising step of the diffusion process, significantly reducing overhead.
  3. Neuron Activation Variation (NAV): Measuring the change in internal neuron activations across layers when a specific token is masked.
  4. Detection: Identifying outliers using a Gaussian distribution threshold to flag malicious triggers based on their unique activation signatures.

Experimental Results

NaviT2I was evaluated against a wide array of benchmarks, including RickBKD, BadT2I, and EvilEdit, using standard datasets like MS-COCO and Flickr30k.

Detection Performance

NaviT2I significantly outperforms existing baselines like T2IShield and UFID. It achieved an average AUROC of 96.3% across various attack types, representing a substantial improvement over the best existing methods.

Detection performance (AUROC %) of NaviT2I compared to state-of-the-art baselines across multiple backdoor attack scenarios.

Efficiency and Generalizability

A key advantage of NaviT2I is its efficiency. By analyzing only the first step, it reduces computational overhead to approximately 7% to 14% of a standard generation cycle. Empirically, it processes samples in ~1.2 seconds, compared to 7.4 seconds for T2IShield and 33 seconds for UFID. Furthermore, the method is architecture-agnostic, maintaining high performance on both UNet and newer Diffusion Transformer (DiT) architectures, where previous baselines often fail.

Conclusion

NaviT2I provides a robust and efficient solution for real-time backdoor detection in T2I models. By leveraging early-step neuron activation signals, it achieves superior detection accuracy with minimal latency. Future work will explore defenses against implicit triggers and more sophisticated adaptive attacks that attempt to mask activation spikes during training.

Summary updated

Abstract

In recent years, text-to-image (T2I) diffusion models have garnered significant attention for their ability to generate high-quality images reflecting text prompts. However, their growing popularity has also led to the emergence of backdoor threats, posing substantial risks. Currently, effective defense strategies against such threats are lacking due to the diversity of backdoor targets in T2I synthesis. In this paper, we propose NaviDet, the first general input-level backdoor detection framework for identifying backdoor inputs across various backdoor targets. Our approach is based on the new observation that trigger tokens tend to induce significant neuron activation variation in the early stage of the diffusion generation process, a phenomenon we term Early-step Activation Variation. Leveraging this insight, NaviDet detects malicious samples by analyzing neuron activation variations caused by input tokens. Through extensive experiments, we demonstrate the effectiveness and efficiency of our method against various T2I backdoor attacks, surpassing existing baselines with significantly lower computational overhead. Furthermore, we rigorously demonstrate that our method remains effective against potential adaptive attacks.
BibTeX
@misc{zhai2025navidet,
  author = {Shengfang Zhai and Jiajun Li and Yue Liu and Huanran Chen and Zhihua Tian and Wenjie Qu and Qingni Shen and Ruoxi Jia and Yinpeng Dong and Jiaheng Zhang},
  title = {{NaviDet: Efficient Input-level Backdoor Detection on Text-to-Image Synthesis via Neuron Activation Variation}},
  year = {2025},
  eprint = {2503.06453},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2503.06453},
}
Figure