DA-Flow: Dual Attention Normalizing Flow for Skeleton-Based Video Anomaly Detection

Ruituo Wu, Yang Chen, Jian Xiao, Bing Li, Jicong Fan, Frédéric Dufaux, Ce Zhu, Yipeng Liu

TL;DR

DA-Flow is a new, ultra-lightweight framework for skeleton-based video anomaly detection that uses a Dual Attention Module (DAM) within a Normalizing Flow architecture. By focusing on the statistical distribution of "normal" human poses rather than just trying to predict the next frame, it achieves state-of-the-art performance with a tiny fraction of the parameters (only 488!) used by competing models.

What Problem Does This Paper Address?

Video Anomaly Detection (VAD) is the task of identifying unusual events (like a fight or a fall) in a video stream. While many systems use raw RGB video, this is computationally expensive and raises privacy concerns. A popular alternative is Skeleton-based VAD (SVAD), which uses only the 2D/3D coordinates of human joints.

However, current SVAD methods—typically based on Graph Convolutional Networks (GCNs) and Temporal Convolutional Networks (TCNs)—suffer from two main issues:

  1. Restricted Receptive Fields: They are great at looking at a single joint or a single moment in time, but they struggle to see the "big picture" (e.g., how a hand relates to a head across several seconds).
  2. Inadequate Cross-Dimension Interaction: They often fail to capture how different dimensions (channels, joints, and time) interact simultaneously.

The authors wanted to build a model that is both "global-aware" and lightweight enough to run on edge devices in real-time.

Methods

The authors introduce DA-Flow, which combines two powerful concepts:

  1. Normalizing Flows (Glow-like): Instead of trying to "reconstruct" a video, the model learns a bijective (reversible) mapping that transforms complex skeleton data into a simple, standard Gaussian distribution. During testing, if a pose sequence has a very low probability under this distribution, it’s flagged as an anomaly.
  2. Dual Attention Module (DAM): This is the "secret sauce" integrated after the GCN layers. It consists of two parallel branches:
  3. Skeleton Attention: Identifies which joints are most crucial by capturing relationships between channels and skeletal joints.
  4. Frame Attention: Identifies which moments in time are most significant by capturing relationships between channels and frames.

By averaging these two perspectives, the model gains a global understanding of the spatio-temporal data without the massive overhead of standard attention mechanisms like Transformers.

Datasets

The model was evaluated on five major benchmarks:

  • ShanghaiTech (STC): A large-scale dataset with 13 different scenes.
  • UBnormal: A synthetic, open-set benchmark with diverse behaviors.
  • UCSD Ped2: A classic dataset focused on pedestrian areas.
  • HR-STC & HR-UBnormal: "Human-Related" subsets that filter out non-human anomalies (like cars on a sidewalk) to focus specifically on skeletal movements.

Key Insights and Ideas

  • The "Zero Training" Surprise: Perhaps the most shocking finding is that the model performs remarkably well even without training. By using random projections (initializing the weights and not updating them), the model still achieved substantial anomaly detection capabilities. This suggests that the statistical structure of skeleton data itself is highly informative for VAD.
  • Efficiency is King: The DAM is specifically designed for skeletal data. While general-purpose computer vision attention modules (like CBAM) are too heavy for this task, DAM provides the necessary global context with almost zero increase in FLOPs.
  • Statistical Modeling vs. Prediction: Most VAD models try to predict the future or reconstruct the past. DA-Flow argues that simply modeling the likelihood of a pose sequence is more robust and requires far fewer parameters.

Key Results

  • Minimalist Parameters: DA-Flow uses only 0.488K parameters. For context, competing models like STG-NF use 0.616K, while others like TrajREC use a massive 4,900K.
  • State-of-the-Art Accuracy: Despite its size, it achieved the highest AUC (Area Under the Curve) scores on several datasets:
  • 86.5% on ShanghaiTech (surpassing the previous SOTA by 0.6%).
  • 74.1% on UBnormal (surpassing the previous SOTA by 2.3%).
  • Robustness: The model maintained high performance even when the input data was "contaminated" with noise or a small percentage of anomalous samples during the training phase.

Limitations and Future Work

The authors noted that performance on the UCSD Ped2 dataset was lower than RGB-based methods. They attribute this to the dataset's low resolution (360x240), which makes it difficult for pose estimators to extract accurate skeletons in the first place.

Future work may involve moving away from the "reconstruction and prediction" paradigm entirely, focusing instead on deeper explorations of the statistical characteristics of normal human motion to further improve detection in low-quality video streams.

Summary updated

Abstract

Cooperation between temporal convolutional networks (TCN) and graph convolutional networks (GCN) as a processing module has shown promising results in skeleton-based video anomaly detection (SVAD). However, to maintain a lightweight model with low computational and storage complexity, shallow GCN and TCN blocks are constrained by small receptive fields and a lack of cross-dimension interaction capture. To tackle this limitation, we propose a lightweight module called the Dual Attention Module (DAM) for capturing cross-dimension interaction relationships in spatio-temporal skeletal data. It employs the frame attention mechanism to identify the most significant frames and the skeleton attention mechanism to capture broader relationships across fixed partitions with minimal parameters and flops. Furthermore, the proposed Dual Attention Normalizing Flow (DA-Flow) integrates the DAM as a post-processing unit after GCN within the normalizing flow framework. Simulations show that the proposed model is robust against noise and negative samples. Experimental results show that DA-Flow reaches competitive or better performance than the existing state-of-the-art (SOTA) methods in terms of the micro AUC metric with the fewest number of parameters. Moreover, we found that even without training, simply using random projection without dimensionality reduction on skeleton data enables substantial anomaly detection capabilities.
BibTeX
@article{wu2025da,
  author = {Ruituo Wu and Yang Chen and Jian Xiao and Bing Li and Jicong Fan and Frédéric Dufaux and Ce Zhu and Yipeng Liu},
  title = {{DA-Flow: Dual Attention Normalizing Flow for Skeleton-Based Video Anomaly Detection}},
  journal = {IEEE Trans. Multimedia},
  year = {2025},
  doi = {10.1109/tmm.2025.3607708},
}

Loading related content...

Figure