SlowFastVAD: Video Anomaly Detection via Integrating Simple Detector and RAG-Enhanced Vision-Language Model

Abstract

SlowFastVAD is a hybrid framework for video anomaly detection (VAD) that mimics human visual perception by combining a lightning-fast "reflex" detector with a sophisticated, reasoning-capable Vision-Language Model (VLM). By using an entropy-based trigger to invoke the VLM only for ambiguous scenes, it achieves state-of-the-art accuracy and provides human-readable explanations without the massive computational cost of traditional VLM inference.

Introduction

Video Anomaly Detection (VAD) involves identifying unusual events in video streams, such as unauthorized vehicles on pedestrian paths or erratic behavior in public spaces. Current VAD systems primarily rely on semi-supervised learning, flagging deviations from a learned "normal" distribution. However, these systems often suffer from high false alarm rates due to rare but normal events and lack explainability—the "black box" problem.

While large Vision-Language Models (VLMs) offer superior reasoning and interpretability, their high computational requirements make them impractical for continuous real-time monitoring. SlowFastVAD resolves this "VLM Paradox" through a dual-pathway architecture inspired by human cognition: a fast, action-oriented response and a slow, deliberate reasoning process.

Expand

Source, p. 2

Methodology

The SlowFastVAD framework integrates two distinct detection branches managed by an intelligent intervention strategy.

Dual-Pathway Architecture

1. Fast Detector: A lightweight Masked Autoencoder (AED-MAE) serves as the primary detector. It processes video at an exceptional 1655 FPS, generating coarse anomaly scores based on reconstruction errors.
2. Slow Detector (RAG-Enhanced VLM): This branch utilizes a VLM (Qwen-VL-Max) augmented with Retrieval-Augmented Generation (RAG). During an initial setup phase, the VLM summarizes "normal" rules for a specific scene. When triggered, it retrieves these rules and uses Chain-of-Thought (CoT) reasoning to provide a definitive classification and a natural language explanation.

Intervention and Fusion

To optimize efficiency, the system calculates the entropy (uncertainty) of the fast detector's output. The "Slow" branch is only activated when entropy exceeds a predefined threshold, indicating ambiguity. Finally, scores from both branches are fused and smoothed using a Gaussian filter to ensure temporal consistency.

Expand

Source, p. 4

Experimental Results

The model was evaluated on four standard VAD benchmarks: UCSD Ped2, CUHK Avenue, ShanghaiTech, and the synthetic UBnormal dataset.

• Accuracy: SlowFastVAD achieved state-of-the-art performance, notably reaching a 99.1% Micro AUC on UCSD Ped2 and 72.2% on the challenging open-set UBnormal dataset.
• Efficiency: By selectively invoking the VLM, the system maintains an operational speed of 16 FPS on a single GPU—32 times faster than a pure VLM approach while simultaneously improving detection accuracy.
• Explainability: Unlike traditional models, SlowFastVAD provides human-readable justifications for its decisions, such as identifying a pedestrian pushing a bicycle as a deviation from expected behavior.

Conclusion

SlowFastVAD demonstrates that integrating high-speed "reflexive" models with high-level reasoning VLMs can overcome the limitations of traditional VAD systems. By leveraging RAG to anchor the VLM to specific environments, the framework reduces hallucinations and provides actionable insights. Future work will focus on further reducing latency for multi-camera deployments and enhancing reasoning for complex temporal dependencies.