From Evaluation to Defense: Advancing Safety in Video Large Language Models

Yiwei Sun, Peiqi Jiang, Chuanbin Liu, Luohao Lin, Zhiying Lu, Hongtao Xie

TL;DR

Integrating video modalities into Large Language Models (LLMs) severely compromises their safety guardrails, causing an average 34.2% drop in defense success rates. To combat this, researchers introduced VideoSafetyEval, a massive real-world safety benchmark, and VideoSafety-R1, a defense framework that uses learnable "alarm tokens" and safety-guided reinforcement learning to shift models from passive harm perception to active safety reasoning.

What Problem Does This Paper Address?

While the safety and alignment of text- and image-based LLMs have been heavily scrutinized, Video LLMs remain a dangerous blind spot. Videos introduce complex temporal dynamics, multi-modal interactions, and evolving contexts that can easily be exploited to bypass text-aligned safety guardrails.

The authors discover a systemic vulnerability: simply adding a video stream to an otherwise aligned text model drastically degrades its ability to refuse harmful prompts. This paper aims to systematically evaluate this multi-modal safety gap and provide a robust post-training defense framework to mitigate it.

Methods

The authors propose VideoSafety-R1, a dual-stage post-training defense framework consisting of two primary innovations:

  1. Alarm Token-Guided Safety Fine-Tuning (AT-SFT): The framework injects lightweight, learnable visual and textual "alarm tokens" into the input sequences. These tokens are trained via multi-task learning to explicitly classify whether the visual or textual components contain harmful elements. This explicitly pre-activates the model's internal defense mechanisms before response generation.
  2. Safety-Guided Group Relative Policy Optimization (GRPO): Moving beyond simple fine-tuning, the authors employ GRPO (a value-model-free reinforcement learning algorithm) to optimize safety reasoning. The model is rewarded for correctly outputting structured <think> chains and dual-modality verification tags (<vidType> and <textType>). A Dynamic Reward Adaptation (DRA) mechanism modulates the weight of language naturalness (ROUGE score) rewards based on whether the model successfully detects harm, forcing strict alignment when a threat is present.

Datasets

The paper introduces two major datasets to the community:

  • VideoSafetyEval (VSE): A large-scale evaluation benchmark containing 11.4k video-query pairs spanning 19 risk categories (e.g., self-harm, cyberbullying, regulated goods) across 10 languages. It features three distinct subsets: VSE-HH (Harmful video, Harmful query), VSE-SH (Safe video, Harmful query), and VSE-SafeQ (Safe queries to test for over-defensiveness/false refusals).
  • VideoSafetyThinking (VST): A post-training dataset containing 46k video-query-thinking response triplets used to bootstrap the framework's chain-of-thought safety reasoning.

The framework's generalization was also validated on existing image safety datasets (MMBench, VLGuard, FigStep) and standard video understanding benchmarks (MVBench, VideoMME).

Figure 2: Framework of VideoSafetyEval (left) and VideoSafetyThinking (right). (a) VideoSafe- tyEval: VSE is generated through filtered video collection, multi-agent-based multimodal annotation, and template-driven query generation. (b) VideoSafetyThinking: VST extends VSE by adding thinking response annotations, serving safety post-training with AT-SFT and Safety-guided GRPO.

Key Insights and Ideas

  • The Video Modality Penalty: Across 21 mainstream Video LLMs, introducing the video modality drops the Defense Success Rate (DSR) by an average of 34.2%.
  • Higher Frame Rates = Lower Safety: Increasing the frames per second (FPS) consistently degrades safety performance. Controlled experiments using static videos stretched across multiple frames proved that this vulnerability is driven by semantic richness rather than token overhead.
  • Fixing the "Attention Sink" Flaw: In standard Vision-Language Models, the first visual token often acts as an ineffective attention sink, absorbing redundant attention that should go to informative visual patches. The authors show that AT-SFT successfully redirects this attention back to the initial text token, reactivating the model's intrinsic textual safety mechanisms.

Key Results

  • Massive Safety Gains: VideoSafety-R1 achieves a 71.1% absolute improvement in Defense Success Rate (DSR) on the adversarial VSE-HH benchmark compared to the baseline VideoLLaMA3-2B model.
  • Cross-Modal Generalization: Despite being trained on video, the defense generalizes remarkably well to static image safety benchmarks, boosting DSR by 59.1% on MMBench, 44.3% on VLGuard, and 15.0% on FigStep.
  • Preserved Utility: The safety alignment does not cause "catastrophic forgetting" of general capabilities; the models showed a negligible average performance drop of just 0.66% across standard video understanding benchmarks like MVBench and VideoMME.
Figure 1: Models’ Defense Success Rate across 19 subcategories. We measure the performance of 17 3B/7B model variants on VSE-HH. Our VideoSafety-R1 attains the highest DSR on 18 subtypes.

Limitations and Future Work

The authors acknowledge several areas for future exploration:

  • Complex/Implicit Attacks: The benchmark primarily covers explicit daily risk scenarios. Future work must address more sophisticated, model-specific adversarial attacks, such as the temporally unsafe use of safety tools.
  • Reasoning Naturalness: Because the safety reasoning chains are generated using template-driven prompts, the model's internal dialogue can sometimes lack the flexibility and fluid nuance of genuine human reasoning.
  • Granular Control: The current alarm tokens and rewards operate at the global video level. Extending these mechanisms to the segment or frame level could provide finer-grained control and better interpretability for long-form video processing.

Summary updated

Abstract

While the safety risks of image-based large language models (Image LLMs) have been extensively studied, their video-based counterparts (Video LLMs) remain critically under-examined. To systematically study this problem, we introduce VideoSafetyEval - a large-scale, real-world benchmark for Video LLM safety, which comprises 11.4k video-query pairs and spans 19 principal risk categories. Based on this, we reveal that integrating video modality degrades safety performance by an average of 34.2%, thereby exposing systemic risks in multimodal attack exploitation. To address this vulnerability, we propose VideoSafety-R1, a dual-stage framework achieving unprecedented safety gains through three innovations: (1) the VideoSafetyThinking dataset contains 46k video-query-thinking response triplets; (2) Alarm Token-Guided Safety Fine-Tuning (AT-SFT) injects learnable alarm tokens into visual and textual sequences, enabling explicit harm perception across modalities via multitask objectives; and (3) safety-guided GRPO enhances defensive reasoning through dynamic policy optimization with rule-based rewards derived from dual-modality verification. These components synergize to shift safety alignment from harm perception to active reasoning. The framework achieves a 71.1% improvement on VSE-HH, and improves by 59.1%, 44.3%, and 15.0% on the image safety datasets MMBench, VLGuard, and FigStep, respectively. Our code and dataset are available at https://github.com/Emiya-syw/VideoSafety-R1.git. Note: This paper contains harmful language and image examples, and reader discretion is recommended.
BibTeX
@inproceedings{sun2026from,
  author = {Yiwei Sun and Peiqi Jiang and Chuanbin Liu and Luohao Lin and Zhiying Lu and Hongtao Xie},
  title = {{From Evaluation to Defense: Advancing Safety in Video Large Language Models}},
  booktitle = {International Conference on Learning Representations},
  year = {2026},
  url = {https://openreview.net/forum?id=QuW5RUDwMo},
}
Figure