Cross-Modality Jailbreak and Mismatched Attacks on Medical Multimodal Large Language Models

Xijie Huang, Xinyuan Wang, Hantao Zhang, Jiawen Xi, Jingkun An, Hao Wang, Chengwei Pan

Introduction

As Medical Multimodal Large Language Models (MedMLLMs) like Med-PaLM and LLaVA-Med move toward clinical deployment, ensuring their safety and security is paramount. While text-based jailbreaks are well-studied, the unique vulnerabilities of multimodal models—which integrate both visual and textual data—remain a critical blind spot. This research identifies a novel vulnerability: clinical mismatches.

The Core Vulnerability: Clinical Mismatches

In clinical settings, data entry errors can lead to mismatches between medical images and their textual descriptions (e.g., a brain MRI mislabeled as a chest CT). The researchers discovered that these semantic inconsistencies can be weaponized to bypass the safety guardrails of MedMLLMs.

The clinical mismatch phenomenon occurs when conflicting image and text inputs disrupt the model's semantic coherence, facilitating a jailbreak.

When a model encounters a mismatch, its internal refusal mechanisms are weakened. This "confusion" allows malicious actors to inject harmful queries that the model would typically reject, such as requests for dangerous medical advice or instructions for illegal activities.

Methodology: 2M-Attack and MCM

The study introduces three key attack frameworks designed to exploit this vulnerability:

  1. 2M-attack (Mismatched Malicious Attack): Exploits natural mismatches by pairing a malicious text query with an unrelated medical image.
  2. O2M-attack (Optimized 2M-attack): Enhances the 2M-attack using optimization techniques to refine adversarial inputs.
  3. MCM (Multimodal Cross-optimization Methodology): The most advanced method, which simultaneously optimizes both text and image modalities. MCM uses a gradient-based approach to add subtle noise to images and select adversarial text tokens, dynamically updating the modality that yields the highest attack effectiveness.
The proposed attack workflow demonstrates how normal queries are transformed into malicious ones through mismatch attributes and optimization.

The 3MAD Dataset

To evaluate these vulnerabilities, the authors created the Multimodal Medical Model Attack Dataset (3MAD). This benchmark includes:

  • 3MAD-66K: Over 66,000 images across 9 modalities and 12 body parts.
  • 3MAD-Tiny-1K: A subset of 6,480 text-image pairs for efficient testing. The data was sourced from major repositories like CheXpert and MIMIC-CXR, ensuring clinical relevance.

Key Results and Transferability

The findings are alarming across multiple state-of-the-art models:

  • High Success Rates: On LLaVA-Med, the MCM method achieved an Attack Success Rate (ASR) of 0.820.
  • Model Vulnerability: Transfer attacks on black-box models showed even higher success; for instance, RadFM reached an ASR of 0.985 under the O2M-attack.
  • Impact of Mismatch: Simply introducing a mismatch (2M-attack) increased the jailbreak success rate by 10% to 20% compared to standard malicious text-only queries.

Conclusion

The research highlights a fundamental flaw in current MedMLLM architectures. Semantic confusion caused by clinical mismatches provides a "backdoor" for jailbreaking even state-of-the-art models. As AI becomes more integrated into healthcare, developing robust defenses—such as input-stage mismatch detection and specialized safety training—is essential to prevent the generation of harmful or unverified medical content.

Summary updated

Abstract

Security concerns related to Large Language Models (LLMs) have been extensively explored, yet the safety implications for Multimodal Large Language Models (MLLMs), particularly in medical contexts (MedMLLMs), remain insufficiently studied. This paper delves into the underexplored security vulnerabilities of MedMLLMs, especially when deployed in clinical environments where the accuracy and relevance of question-and-answer interactions are critically tested against complex medical challenges. By combining existing clinical medical data with atypical natural phenomena, we define the mismatched malicious attack (2M-attack) and introduce its optimized version, known as the optimized mismatched malicious attack (O2M-attack or 2M-optimization). Using the voluminous 3MAD dataset that we construct, which covers a wide range of medical image modalities and harmful medical scenarios, we conduct a comprehensive analysis and propose the MCM optimization method, which significantly enhances the attack success rate on MedMLLMs. Evaluations with this dataset and attack methods, including white-box attacks on LLaVA-Med and transfer attacks (black-box) on four other SOTA models, indicate that even MedMLLMs designed with enhanced security features remain vulnerable to security breaches. Our work underscores the urgent need for a concerted effort to implement robust security measures and enhance the safety and efficacy of open-source MedMLLMs, particularly given the potential severity of jailbreak attacks and other malicious or clinically significant exploits in medical settings. Our code is available at https://github.com/dirtycomputer/O2M_attack.
BibTeX
@inproceedings{huang2025cross,
  author = {Xijie Huang and Xinyuan Wang and Hantao Zhang and Jiawen Xi and Jingkun An and Hao Wang and Chengwei Pan},
  title = {{Cross-Modality Jailbreak and Mismatched Attacks on Medical Multimodal Large Language Models}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i4.32396},
}
Figure