Abstract
Traditional Video Anomaly Detection (VAD) focuses primarily on identifying whether a frame is normal or abnormal. However, these models often fail to provide the contextual reasoning necessary for complex decision-making. This paper introduces Vad-R1, an end-to-end framework designed for Video Anomaly Reasoning (VAR). By employing a novel Perception-to-Cognition Chain-of-Thought (P2C-CoT) and a self-verifying reinforcement learning algorithm (AVA-GRPO), Vad-R1 moves beyond simple detection to explain the underlying causes and consequences of anomalous events. Experimental results demonstrate that Vad-R1 achieves state-of-the-art performance, significantly outperforming proprietary models such as GPT-4o and o4-mini in both detection accuracy and reasoning quality.
1. Introduction
Video Anomaly Detection (VAD) is a cornerstone of modern surveillance and autonomous driving systems. While traditional models are effective at flagging statistical outliers, they lack the cognitive depth to explain why an event is considered abnormal. Recent advancements in Multimodal Large Language Models (MLLMs) have enabled text-based descriptions of anomalies, but these descriptions remain largely superficial.
To address this, we define the task of Video Anomaly Reasoning (VAR). VAR requires a model to not only detect an event but also to perform step-by-step reasoning regarding the environment, social norms, and potential outcomes. We present Vad-R1, a model specifically optimized for this perception-to-cognition transition.
2. The Vad-R1 Framework
2.1 Perception-to-Cognition Chain-of-Thought (P2C-CoT)
The core of Vad-R1 is a structured reasoning process called P2C-CoT. Unlike generic "think step-by-step" prompts, P2C-CoT enforces a four-stage cognitive pipeline:
- Global Perception: The model observes the overall scene and establishes a baseline for the "normal" environment.
- Local Perception: The model identifies specific suspicious clips, objects, or temporal segments.
- Shallow Cognition: The model identifies the specific anomaly and provides evidence based on visual cues.
- Deep Cognition: The model reasons about underlying causes, the violation of social norms, and the likely consequences of the event.
2.2 Two-Stage Training and AVA-GRPO
Vad-R1 is trained in two distinct phases to ensure both logical structure and predictive accuracy.
Stage 1: Supervised Fine-Tuning (SFT). The model is trained on high-quality, dense P2C-CoT annotations. This stage is critical for teaching the model the specific format and logic required for deep reasoning.
Stage 2: Reinforcement Learning (RL) via AVA-GRPO. To refine the model without requiring expensive dense annotations, we introduce Anomaly Verification Augmented Group Relative Policy Optimization (AVA-GRPO). This algorithm utilizes weak, video-level labels (Normal/Abnormal) and a self-verification mechanism. If the model identifies an anomaly in a specific temporal segment, the system trims that segment and re-evaluates the remaining video. If the model then correctly identifies the remaining video as "Normal," it confirms the logical consistency of the original prediction and receives a positive reward. This process also penalizes "temporal hacking," where models attempt to guess based on frame position rather than content.
3. Experimental Setup
3.1 The Vad-Reasoning Dataset
We introduce the Vad-Reasoning dataset, comprising 8,641 videos sourced from established VAD benchmarks (e.g., UCF-Crime, XD-Violence) and the internet. The dataset is divided into:
- Vad-Reasoning-SFT: 1,755 videos with dense, structured CoT annotations generated via a multi-stage pipeline.
- Vad-Reasoning-RL: 6,448 videos with binary labels for reinforcement learning.
3.2 Benchmarking
Performance was evaluated on the Vad-Reasoning test set and the VANE Benchmark, which includes both real-world and AI-generated anomalies for zero-shot evaluation.
4. Results and Discussion
4.1 Detection and Reasoning Performance
Vad-R1 demonstrates superior performance across all primary metrics. On the Vad-Reasoning test set, it achieved an accuracy of 87.5% and an F1 score of 86.2%, significantly higher than GPT-4o (71.1% accuracy).
In terms of reasoning quality, Vad-R1 achieved a BLEU-2 score of 0.233, outperforming specialized reasoning models like Gemini 2.5 Pro (0.145) and o4-mini (0.106). LLM-guided evaluations for "Reasonability" and "Consistency" further confirmed that Vad-R1's explanations are more logically sound and factually aligned with visual evidence than its competitors.
4.2 Key Insights
- Structure is Essential: Structured P2C-CoT is necessary for performance gains; unstructured reasoning prompts do not yield the same benefits.
- SFT as a Foundation: RL is most effective when the model has already learned the reasoning format through SFT.
- Self-Verification: AVA-GRPO successfully bridges the gap between cheap, weak labels and complex reasoning tasks.
5. Conclusion and Limitations
Vad-R1 represents a significant step toward autonomous systems that can understand and explain complex human environments. While the model sets a new state-of-the-art, its primary limitation is inference speed. The multi-step CoT process increases token generation, creating computational overhead. Future research will focus on accelerating this reasoning process to make it suitable for real-time surveillance applications.