TL;DR
Integrating video modalities into Large Language Models (LLMs) severely compromises their safety guardrails, causing an average 34.2% drop in defense success rates. To combat this, researchers introduced VideoSafetyEval, a massive real-world safety benchmark, and VideoSafety-R1, a defense framework that uses learnable "alarm tokens" and safety-guided reinforcement learning to shift models from passive harm perception to active safety reasoning.
What Problem Does This Paper Address?
While the safety and alignment of text- and image-based LLMs have been heavily scrutinized, Video LLMs remain a dangerous blind spot. Videos introduce complex temporal dynamics, multi-modal interactions, and evolving contexts that can easily be exploited to bypass text-aligned safety guardrails.
The authors discover a systemic vulnerability: simply adding a video stream to an otherwise aligned text model drastically degrades its ability to refuse harmful prompts. This paper aims to systematically evaluate this multi-modal safety gap and provide a robust post-training defense framework to mitigate it.
Methods
The authors propose VideoSafety-R1, a dual-stage post-training defense framework consisting of two primary innovations:
- Alarm Token-Guided Safety Fine-Tuning (AT-SFT): The framework injects lightweight, learnable visual and textual "alarm tokens" into the input sequences. These tokens are trained via multi-task learning to explicitly classify whether the visual or textual components contain harmful elements. This explicitly pre-activates the model's internal defense mechanisms before response generation.
- Safety-Guided Group Relative Policy Optimization (GRPO): Moving beyond simple fine-tuning, the authors employ GRPO (a value-model-free reinforcement learning algorithm) to optimize safety reasoning. The model is rewarded for correctly outputting structured
<think>chains and dual-modality verification tags (<vidType>and<textType>). A Dynamic Reward Adaptation (DRA) mechanism modulates the weight of language naturalness (ROUGE score) rewards based on whether the model successfully detects harm, forcing strict alignment when a threat is present.
Datasets
The paper introduces two major datasets to the community:
- VideoSafetyEval (VSE): A large-scale evaluation benchmark containing 11.4k video-query pairs spanning 19 risk categories (e.g., self-harm, cyberbullying, regulated goods) across 10 languages. It features three distinct subsets:
VSE-HH(Harmful video, Harmful query),VSE-SH(Safe video, Harmful query), andVSE-SafeQ(Safe queries to test for over-defensiveness/false refusals). - VideoSafetyThinking (VST): A post-training dataset containing 46k video-query-thinking response triplets used to bootstrap the framework's chain-of-thought safety reasoning.
The framework's generalization was also validated on existing image safety datasets (MMBench, VLGuard, FigStep) and standard video understanding benchmarks (MVBench, VideoMME).
Key Insights and Ideas
- The Video Modality Penalty: Across 21 mainstream Video LLMs, introducing the video modality drops the Defense Success Rate (DSR) by an average of 34.2%.
- Higher Frame Rates = Lower Safety: Increasing the frames per second (FPS) consistently degrades safety performance. Controlled experiments using static videos stretched across multiple frames proved that this vulnerability is driven by semantic richness rather than token overhead.
- Fixing the "Attention Sink" Flaw: In standard Vision-Language Models, the first visual token often acts as an ineffective attention sink, absorbing redundant attention that should go to informative visual patches. The authors show that AT-SFT successfully redirects this attention back to the initial text token, reactivating the model's intrinsic textual safety mechanisms.
Key Results
- Massive Safety Gains: VideoSafety-R1 achieves a 71.1% absolute improvement in Defense Success Rate (DSR) on the adversarial VSE-HH benchmark compared to the baseline VideoLLaMA3-2B model.
- Cross-Modal Generalization: Despite being trained on video, the defense generalizes remarkably well to static image safety benchmarks, boosting DSR by 59.1% on MMBench, 44.3% on VLGuard, and 15.0% on FigStep.
- Preserved Utility: The safety alignment does not cause "catastrophic forgetting" of general capabilities; the models showed a negligible average performance drop of just 0.66% across standard video understanding benchmarks like MVBench and VideoMME.
Limitations and Future Work
The authors acknowledge several areas for future exploration:
- Complex/Implicit Attacks: The benchmark primarily covers explicit daily risk scenarios. Future work must address more sophisticated, model-specific adversarial attacks, such as the temporally unsafe use of safety tools.
- Reasoning Naturalness: Because the safety reasoning chains are generated using template-driven prompts, the model's internal dialogue can sometimes lack the flexibility and fluid nuance of genuine human reasoning.
- Granular Control: The current alarm tokens and rewards operate at the global video level. Extending these mechanisms to the segment or frame level could provide finer-grained control and better interpretability for long-form video processing.
Summary updated