Sparse Reasoning is Enough: Biological-Inspired Framework for Video Anomaly Detection with Large Pre-trained Models

He Huang, Zixuan Hu, Dongxiao Li, Yao Xiao, Ling-Yu DuanarXiv 2025 · DOI · arXiv

Abstract

Video Anomaly Detection (VAD) has recently been advanced by the reasoning capabilities of Large Vision-Language Models (LVLMs). However, the computational overhead of dense reasoning—processing every frame with high-parameter models—remains a significant barrier to real-world deployment. We present ReCoVAD, a training-free, bio-inspired framework that mimics the human nervous system's dual-pathway architecture. By utilizing a lightweight "Reflex Pathway" to filter redundant data and a "Conscious Pathway" to process novel visual input, ReCoVAD achieves state-of-the-art performance while processing as few as 16% of video frames compared to traditional dense-inference methods.

1. Introduction

Identifying anomalous events such as accidents, fights, or thefts in surveillance footage is a critical task in computer vision. While Large Visual Language Models (LVLMs) offer unprecedented reasoning capabilities for this task, they impose a heavy "compute tax." Most current state-of-the-art systems perform dense reasoning, running multi-billion parameter models on every single frame to ensure no anomaly is missed. This approach is often redundant, as many frames in a surveillance stream contain repetitive, normal behavior.

ReCoVAD addresses this efficiency gap by asking a fundamental question: Is dense reasoning necessary, or can we achieve comparable results by "thinking" only when something novel occurs? Inspired by the biological efficiency of the human nervous system, ReCoVAD introduces a sparse reasoning paradigm that balances high-level cognition with low-level filtering.

Conceptual overview of the bio-inspired ReCoVAD system compared to the human nervous system.

2. Methodology

The ReCoVAD framework is structured around a dual-pathway architecture that separates routine monitoring from intensive analysis.

2.1 The Reflex Pathway (Bottom-Up)

This lightweight component acts as the system's first line of defense. Utilizing a CLIP-based encoder, it maps video frames into "decision vectors" based on a predefined set of text prototypes (e.g., "normal walking" vs. "car crash"). The pathway maintains a dynamic memory of past frames. If a incoming frame is semantically similar to an entry in memory, the system "reflexively" assigns it a score without activating the more resource-intensive models.

2.2 The Conscious Pathway (Top-Down)

When a frame is identified as "novel"—meaning it does not match the current memory—it is escalated to the Conscious Pathway. Here, a medium-scale LVLM (Qwen2.5-VL-7B) generates a detailed textual description and an anomaly score. This pathway provides the deep reasoning necessary to interpret complex or ambiguous scenes.

2.3 Closed-Loop Feedback

To ensure the system evolves, a high-level LLM (DeepSeek-V3) periodically reviews accumulated descriptions. This feedback loop refines the text prototypes and corrects past misclassifications, allowing the Reflex Pathway to become more context-aware over time and better filter redundant information.

Detailed architecture of the Reflex and Conscious pathways within the ReCoVAD system.

3. Experimental Results

ReCoVAD was evaluated on two major benchmarks for real-world anomaly detection: UCF-Crime (1,900 videos across 13 anomaly types) and XD-Violence (4,754 videos focusing on violent events).

3.1 Accuracy and Efficiency

ReCoVAD outperformed previous training-free methods while utilizing a fraction of the computational resources:

  • UCF-Crime: Achieved a state-of-the-art 82.28% AUC, processing only 28.55% of the frames required by dense methods.
  • XD-Violence: Achieved 86.38% AUC, processing a mere 16.04% of frames.

Compared to the previous leading method, LAVAD, ReCoVAD showed a ~2% improvement in AUC while being significantly faster and more cost-effective to operate.

4. Discussion and Future Work

The primary insight of this work is that sparse reasoning is sufficient for high-accuracy VAD. By focusing "conscious" effort only on novel visual data, the model maintains performance while ignoring the noise of repetitive normal behavior.

However, some limitations remain. The system's performance is currently tied to the specific LVLM/LLM combinations used, and its reliance on text prototypes may pose challenges for highly niche or abstract anomalies. Future research will focus on investigating model generalization and deploying the framework on edge devices, where the benefits of sparse reasoning are most impactful.

Summary updated

BibTeX
@misc{huang2025sparse,
  author = {He Huang and Zixuan Hu and Dongxiao Li and Yao Xiao and Ling-Yu Duan},
  title = {{Sparse Reasoning is Enough: Biological-Inspired Framework for Video Anomaly Detection with Large Pre-trained Models}},
  year = {2025},
  eprint = {2511.17094},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2511.17094},
}
Figure