TVChain: Leveraging Textual-Visual Prompt Chains for Jailbreaking Large Vision-Language Models

Hao Yu, Ke Liang, Junxian Duan, Jun Wang, Siwei Wang, Chuan Ma, Xinwang Liu

Abstract

Large Vision-Language Models (LVLMs) enhance the capabilities of Large Language Models by integrating visual inputs, thereby enabling advanced multimodal reasoning across diverse applications. However, these enhanced reasoning capabilities introduce new security risks, particularly to jailbreaking attacks that bypass built-in safety mechanisms to elicit harmful or unauthorized outputs. In this work, we propose TVChain, a novel black-box jailbreaking framework that explicitly intervenes in both the visual and textual reasoning processes of LVLMs. TVChain decomposes malicious prompts into a sequence of semantically meaningful sub-images that represent relevant objects and behaviors, thereby circumventing direct exposure of illicit content. In parallel, a carefully designed chain-of-thought (CoT) textual prompt is employed to steer the model's reasoning toward reconstructing the intended activity in a covert yet effective manner. Extensive evaluations on eleven LVLMs across two benchmark datasets and three state-of-the-art defenses validate the effectiveness and robustness of TVChain.

Introduction

As Large Vision-Language Models (LVLMs) become more sophisticated, they integrate visual and textual reasoning to solve complex tasks. However, this multimodal integration opens new security backdoors. Current "jailbreaking" attempts—which try to trick a model into generating prohibited content (e.g., instructions for hacking or violence)—typically rely on adversarial images (adding noise) or typographic attacks (embedding text).

These methods suffer from a "smoking gun" problem: they concentrate the harmful intent into a single image, making them easy for safety filters like Llama Guard to detect. Furthermore, many existing attacks require "white-box" access to model weights, which is impractical for commercial APIs like GPT-4o.

Methodology: The TVChain Framework

TVChain introduces a "divide and conquer" strategy called Textual-Visual Prompt Chains. Instead of presenting a single malicious image, it uses a sequence of neutral components to bypass safety heuristics.

Visual Prompt Chain

The framework first extracts core objects and behaviors from a malicious query using an LLM. For example, a query about "hacking a bank account" is decomposed into elements like "a hacker," "a bank server," and "a person typing." A text-to-image model then generates individual sub-images for each element, which are arranged into a composite grid. Individually, these images appear benign; the harmful intent only emerges when the LVLM views them as a sequence.

Comparison of visual jailbreaking strategies: (a) MM-Safety and (b) HADES use concentrated harm, while (c) TVChain distributes intent across a grid of benign sub-images.

Textual Prompt Chain

To ensure the LVLM connects these visual dots, TVChain employs a 5-step Chain-of-Thought (CoT) scaffold:

  1. Observe: Examine each panel in the grid.
  2. Identify: Determine the actors and actions involved.
  3. Link: Connect the actions across panels.
  4. Infer Goal: Deduce the overall purpose of the sequence.
  5. Output: Provide the final detailed guide based on the inferred goal.

By following this disciplined reasoning path, the model effectively "talks itself" into the jailbreak, bypassing the abrupt semantic leaps that usually trigger safety filters.

Key Results and Evaluation

TVChain was evaluated against 11 models (7 open-source, 4 commercial) using the HADES and MM-Safety benchmarks. The results demonstrate that the very reasoning capabilities that make LVLMs powerful are the exact mechanisms TVChain exploits.

  • High Attack Success Rates (ASR): On the HADES dataset, TVChain achieved an ASR of over 90% on several open-source models. Commercial models were also vulnerable, with GPT-4o seeing an ASR of 74.13% and Gemini 2.0 FT reaching 82.27%.
  • Evasion of Defenses: TVChain proved significantly harder to detect than traditional attacks. Against Llama Guard 3, TVChain's detection rate was markedly lower than that of the HADES baseline across all categories.
Detection success rates for TVChain across different harm categories and models, showing significantly lower detection compared to concentrated-harm attacks.

Conclusion

TVChain demonstrates that distributed harm is far stealthier than concentrated harm. By spreading malicious intent across a grid of sub-images and leveraging the model's own reasoning to reconstruct that intent, the attack stays below the detection threshold of state-of-the-art defenses. Future safety filters must evolve to perform "compositional" safety checks that look for harmful patterns across sequences of images rather than analyzing them in isolation.

Summary updated

Abstract

Large Vision-Language Models (LVLMs) enhance the capabilities of Large Language Models by integrating visual inputs, thereby enabling advanced multimodal reasoning across diverse applications. However, these enhanced reasoning capabilities introduce new security risks, particularly to jailbreaking attacks that bypass built-in safety mechanisms to elicit harmful or unauthorized outputs. While recent efforts have explored adversarial and typographic prompts, most existing attacks suffer from three key limitations: reliance on auxiliary models, limited effectiveness in black-box scenarios, and inadequate exploitation of the LVLMs' intrinsic reasoning abilities. In this work, we propose TVChain, a novel black-box jailbreaking framework that explicitly intervenes in both the visual and textual reasoning processes of LVLMs. TVChain decomposes malicious prompts into a sequence of semantically meaningful sub-images that represent relevant objects and behaviors, thereby circumventing direct exposure of illicit content. In parallel, a carefully designed chain-of-thought (CoT) textual prompt is employed to steer the model's reasoning toward reconstructing the intended activity in a covert yet effective manner. We demonstrate that this compositional prompting strategy reduces the likelihood of triggering safety mechanisms while preserving attack efficacy. Extensive evaluations on eleven LVLMs (seven open-source and four commercial) across two benchmark datasets and three state-of-the-art defenses validate the effectiveness and robustness of TVChain.
BibTeX
@inproceedings{yu2026tvchain,
  author = {Hao Yu and Ke Liang and Junxian Duan and Jun Wang and Siwei Wang and Chuan Ma and Xinwang Liu},
  title = {{TVChain: Leveraging Textual-Visual Prompt Chains for Jailbreaking Large Vision-Language Models}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2026},
  url = {https://openreview.net/forum?id=MVKF4ESBwe},
}
Figure