Abstract
Cooperation between temporal convolutional networks (TCN) and graph convolutional networks (GCN) as a processing module has shown promising results in skeleton-based video anomaly detection (SVAD). However, to maintain a lightweight model with low computational and storage complexity, shallow GCN and TCN blocks are constrained by small receptive fields and a lack of cross-dimension interaction capture. To tackle this limitation, we propose a lightweight module called the Dual Attention Module (DAM) for capturing cross-dimension interaction relationships in spatio-temporal skeletal data. It employs the frame attention mechanism to identify the most significant frames and the skeleton attention mechanism to capture broader relationships across fixed partitions with minimal parameters and flops. Furthermore, the proposed Dual Attention Normalizing Flow (DA-Flow) integrates the DAM as a post-processing unit after GCN within the normalizing flow framework. Simulations show that the proposed model is robust against noise and negative samples. Experimental results show that DA-Flow reaches competitive or better performance than the existing state-of-the-art (SOTA) methods in terms of the micro AUC metric with the fewest number of parameters. Moreover, we found that even without training, simply using random projection without dimensionality reduction on skeleton data enables substantial anomaly detection capabilities.
Introduction
Video Anomaly Detection (VAD) is critical for identifying unusual events like accidents or security breaches in surveillance streams. While RGB-based methods are common, they are computationally intensive and pose privacy risks. Skeleton-based VAD (SVAD) offers a privacy-preserving, lightweight alternative by focusing on human joint coordinates. However, existing SVAD architectures often struggle with restricted receptive fields and inadequate cross-dimension interaction, failing to capture how different joints relate to one another across time. DA-Flow addresses these challenges by combining Normalizing Flows with a novel Dual Attention Module (DAM).
The DA-Flow Framework
DA-Flow shifts the paradigm from reconstruction-based or predictive modeling to statistical likelihood estimation. Instead of trying to predict the next frame, the model learns a bijective (reversible) mapping that transforms complex skeleton data into a standard Gaussian distribution.
As shown in the architecture above, the framework processes pose sequences through a series of flow blocks. Each block splits the input features, processes one half through a GCN followed by the Dual Attention Module (DAM), and then recombines them. During inference, any pose sequence that yields a low probability under the learned Gaussian distribution is flagged as an anomaly.
Dual Attention Module (DAM)
The DAM is the core innovation designed to capture global spatio-temporal context without the heavy overhead of Transformers. It operates through two parallel branches:
- Skeleton Attention: Identifies crucial joints by capturing relationships between channels and skeletal joints.
- Frame Attention: Identifies significant moments in time by capturing relationships between channels and frames.
By integrating these perspectives, the model achieves a global understanding of the data with negligible increases in FLOPs or parameters.
Key Results
DA-Flow was evaluated on major benchmarks including ShanghaiTech (STC), UBnormal, and UCSD Ped2. The model demonstrated exceptional efficiency, utilizing only 0.488K parameters—a fraction of the size of competing models like TrajREC (4,900K).
As illustrated in the figure above, DA-Flow occupies the top-left quadrant, representing the ideal balance of high accuracy and low complexity. Key performance highlights include:
- 86.5% AUC on ShanghaiTech (surpassing previous SOTA by 0.6%).
- 74.1% AUC on UBnormal (surpassing previous SOTA by 2.3%).
One of the most surprising findings was the model's performance with zero training. By using random projections (initializing weights without updating them), the model still achieved substantial detection capabilities, suggesting that the inherent statistical structure of skeleton data is highly informative for anomaly detection.
Conclusion
DA-Flow proves that minimalist AI can achieve state-of-the-art results in video anomaly detection. By focusing on statistical likelihood through Normalizing Flows and employing a lightweight Dual Attention Module, the framework provides a robust, real-time solution suitable for edge devices. While low-resolution video remains a challenge for pose estimation accuracy, the transition toward likelihood-based modeling offers a promising path for future SVAD research.
Summary updated