REVEAL: Multi-turn Evaluation of Image-Input Harms for Vision LLMs

Madhur Jindal, Saurabh DeshpandeIJCAI 2025 · OpenReview

TL;DR

Vision-Language Models (VLLMs) are powerful but their safety is hard to evaluate with traditional methods. This paper introduces REVEAL, a new framework for automatically testing VLLMs for harmful content in multi-turn, image-infused conversations, revealing deeper vulnerabilities than previously known.

What Problem Does This Paper Address?

Large Language Models (LLMs) have revolutionized AI, and Vision LLMs (VLLMs) are the next frontier, combining text and image understanding. This integration opens up exciting new applications but also introduces complex safety and ethical challenges. Current safety evaluation methods, often designed for text-only, single-turn interactions, are insufficient for VLLMs. They fail to capture the nuanced ways harm can emerge in dynamic, multi-modal conversations where images and text interact. This gap makes it difficult to ensure VLLMs are safe and responsible, especially as they become more pervasive.

Methods

The core contribution is the REVEAL (Responsible Evaluation of Vision-Enabled AI LLMs) Framework. This is a scalable, automated pipeline designed to uncover image-input harms in VLLMs. REVEAL works in several stages:

  1. Image Mining: It sources real-world images relevant to specific harm policies (e.g., violence, sexual harm, misinformation).
  2. Image Search Query Generation: It creates targeted search queries to find appropriate images.
  3. Seed Set Generation: It generates initial "seed" conversational turns that are relevant to both the harm policy and the mined image.
  4. Conversational Expansion: This is a key innovation. It uses a "crescendo attack" strategy to expand single-turn seeds into multi-turn conversations. This means starting with innocuous topics and gradually intensifying the conversation to elicit harmful responses, mimicking realistic user interactions.
  5. Evaluation: The generated multi-turn conversations are then fed to the VLLMs, and an evaluator (GPT-4o in this case, validated by human experts) assesses the responses against the defined harm policies.

The framework is designed to be modular and extensible, allowing for new harm categories and VLLMs to be easily integrated.

Datasets

The paper introduces a newly generated multi-turn adversarial dataset specifically for evaluating image-input harms in VLLMs. This dataset is created using the REVEAL framework's pipeline. It is designed to be more realistic than static benchmarks and is made publicly available to encourage further research.

Key Insights and Ideas

  • Multi-turn interactions are significantly more dangerous: VLLMs exhibit much higher defect rates in multi-turn conversations compared to single-turn interactions, revealing deeper vulnerabilities.
  • Misinformation is a critical blind spot: Models struggle to handle nuanced misinformation attacks that require contextual understanding, especially in multi-turn settings.
  • Safety vs. Usability Trade-off: Models often prioritize conversational flow over strict safety protocols in multi-turn settings, leading to lower refusal rates but higher defect rates.
  • Model size isn't a guarantee of safety: While larger models generally perform better, they are not immune to multi-turn harms.
  • Harm categories behave differently: Violence is a high-defect area, while sexual harm shows relative robustness in multi-turn scenarios.

Key Results

The study evaluated five state-of-the-art VLLMs: GPT-4o, Llama-3.2, Qwen2-VL, Phi3.5V, and Pixtral.

  • Overall Defect Rates: Multi-turn (MT) conversations showed significantly higher defect rates (11.40%) compared to single-turn (ST) (5.78%).
  • Overall Refusal Rates: MT conversations had lower refusal rates (5.82%) than ST (19.47%).
  • Model Performance:
    • GPT-4o showed the most balanced performance, with a Safety-Usability Index (SUI) of 1.61% in MT.
  • Pixtral also performed well, with an SUI of 1.70% in MT.
  • Llama-3.2 had the highest MT defect rate at 16.55%.
  • Qwen2-VL had the highest MT refusal rate at 19.1%.
  • Phi3.5V showed a large increase in MT defect rate over ST.
  • Harm Category Breakdown: Violence had the highest defect rates in ST, while misinformation's defect rate increased significantly in MT. Sexual harm showed relative robustness in MT.
  • Text-Only vs. Multi-Modal: GPT-4o's text-only evaluations showed consistently lower defect rates, suggesting that the multi-modal aspect introduces specific vulnerabilities.

Limitations and Future Work

  • Language Limitation: The current framework and experiments are limited to English.
  • Image Sourcing: The framework relies on mining existing images and does not explore image generation as a source of adversarial content.
  • Model Compliance: The generation of harmful content for evaluation relies on models being capable of complying with such requests, which could affect the effectiveness of the evaluation.

Future work could involve extending the framework to other languages, incorporating image generation techniques, and exploring more diverse harm categories. The authors emphasize the modularity of REVEAL, making it adaptable for ongoing safety evaluations as VLLM technology evolves.

Summary updated

Abstract

Vision Large Language Models (VLLMs) represent a significant advancement in artificial intelligence by integrating image-processing capabilities with textual understanding, thereby enhancing user interactions and expanding application domains. However, their increased complexity introduces novel safety and ethical challenges, particularly in multi-modal and multi-turn conversations. Traditional safety evaluation frameworks, designed for text-based, single-turn interactions, are inadequate for addressing these complexities. To bridge this gap, we introduce the REVEAL (Responsible Evaluation of Vision-Enabled AI LLMs) Framework, a scalable and automated pipeline for evaluating image-input harms in VLLMs. REVEAL includes automated image mining, synthetic adversarial data generation, multi-turn conversational expansion using crescendo attack strategies, and comprehensive harm assessment through evaluators like GPT-4o. We extensively evaluated five state-of-the-art VLLMs, GPT-4o, Llama-3.2, Qwen2-VL, Phi3.5V, and Pixtral, across three important harm categories: sexual harm, violence, and misinformation. Our findings reveal that multi-turn interactions result in significantly higher defect rates compared to single-turn evaluations, highlighting deeper vulnerabilities in VLLMs. Notably, GPT-4o demonstrated the most balanced performance as measured by our Safety-Usability Index (SUI) followed closely by Pixtral. Additionally, misinformation emerged as a critical area requiring enhanced contextual defenses. Llama-3.2 exhibited the highest MT defect rate (16.55%) while Qwen2-VL showed the highest MT refusal rate (19.1%).
BibTeX
@inproceedings{jindal2025reveal,
  author = {Madhur Jindal and Saurabh Deshpande},
  title = {{REVEAL: Multi-turn Evaluation of Image-Input Harms for Vision LLMs}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2025},
  url = {https://openreview.net/forum?id=NuskGiKFUh},
}
Figure