Crowd-Level Abnormal Behavior Detection via Multi-Scale Motion Consistency Learning

Linbo Luo, Yuanjing Li, Haiyan Yin, Shangwei Xie, Ruimin Hu, Wentong Cai

Abstract

Traditional video anomaly detection (VAD) often focuses on individual-level actions, failing to capture dangerous collective phenomena like crowd turbulence or surges. This paper presents MSMC-Net, a framework designed to detect Crowd-Level Abnormal Behaviors (CABs) by modeling motion consistency across multiple spatial scales. Using Graph Convolutional Networks (GCNs) and a multi-scale fusion mechanism with self-attention, the model identifies deviations from normal collective motion patterns. Evaluated on benchmarks including the UMN, Hajj, and Love Parade datasets, MSMC-Net demonstrates superior performance, particularly in high-density real-world disaster scenarios.

Introduction

Most Video Anomaly Detection (VAD) systems are designed to identify "salient" differences—an individual performing an unusual action, such as fighting or theft. However, the most catastrophic crowd events, such as the 2010 Love Parade disaster, involve Crowd-Level Abnormal Behaviors (CABs) that emerge from collective dynamics rather than individual deviance.

CABs present two primary challenges for detection:

  1. Local Invisibility: At an individual level, motion may appear normal. The anomaly only becomes apparent at a macro-scale when observing the collective.
  2. Scale Variation: Crowd densities vary significantly across different environments (e.g., a hallway vs. a public square), making fixed-scale models ineffective.

The authors address these challenges by shifting the focus from individual actions to the consistency of collective motion across multiple spatial scales.

Methodology: MSMC-Net

The proposed Multi-Scale Motion Consistency Network (MSMC-Net) operates through a three-phase pipeline: motion consistency representation, graph encoding, and multi-scale fusion.

The MSMC-Net architecture, featuring GCN encoding and multi-scale fusion decoding.

Motion Consistency Representation

Instead of processing raw pixels, the model calculates motion consistency using optical flow to measure:

  • Spatial Consistency: Similarity of movement within a region (inner-consistency) and between neighboring regions (inter-consistency).
  • Temporal Consistency: Velocity changes over time, which helps identify "stop-and-go" waves or sudden turbulence.

Graph Encoding and Multi-Scale Fusion

These metrics are mapped onto a graph where regions are nodes and relationships are edges. Graph Convolutional Networks (GCNs) capture structural patterns. To handle scale variation, the model extracts features at multiple scales (e.g., 1x, 2x, and 4x pedestrian size). A self-attention mechanism adaptively fuses these scales into a single representation used to reconstruct the crowd's motion state. An auxiliary decoding task ensures the network learns robust features at every scale individually.

Experimental Evaluation

The study utilizes three large-scale datasets: the standard UMN benchmark for escape behaviors, the Hajj dataset (surveillance of the annual pilgrimage), and the Love Parade dataset (real-world footage from the 2010 disaster).

Results

MSMC-Net significantly outperformed state-of-the-art (SOTA) methods across all benchmarks. On the UMN dataset, it achieved an AUC of 94.4%. Most notably, on the Love Parade dataset—where previous SOTA methods struggled to exceed 60% AUC—MSMC-Net reached 82.2% AUC, demonstrating its effectiveness in high-density disaster scenarios.

Visualization of anomaly scores over time, showing a sharp spike during an escape event.

Discussion and Conclusion

The core insight of this work is that high motion consistency serves as a proxy for safety, while low consistency indicates risk (turbulence, opposing flows). The multi-scale approach is essential; for instance, the Love Parade's high density requires a 4x scale to capture the "big picture" of turbulence that smaller scales miss. While effective, future work remains in cross-scenario generalization and real-time optimization for smart city deployments. Furthermore, by relying on motion vectors rather than facial features, the method offers inherent privacy advantages for public surveillance.

Summary updated

Abstract

Detecting abnormal crowd motion emerging from complex interactions of individuals is paramount to ensure the safety of crowds. Crowd-level abnormal behaviors (CABs), e.g., counter flow and crowd turbulence, are proven to be the crucial causes of many crowd disasters. In the recent decade, video anomaly detection (VAD) techniques have achieved remarkable success in detecting individual-level abnormal behaviors (e.g., sudden running, fighting and stealing), but research on VAD for CABs is rather limited. Unlike individual-level anomaly, CABs usually do not exhibit salient difference from the normal behaviors when observed locally, and the scale of CABs could vary from one scenario to another. In this paper, we present a systematic study to tackle the important problem of VAD for CABs with a novel crowd motion learning framework, multi-scale motion consistency network (MSMC-Net). MSMC-Net first captures the spatial and temporal crowd motion consistency information in a graph representation. Then, it simultaneously trains multiple feature graphs constructed at different scales to capture rich crowd patterns. An attention network is used to adaptively fuse the multi-scale features for better CAB detection. For the empirical study, we consider three large-scale crowd event datasets, UMN, Hajj and Love Parade. Experimental results show that MSMC-Net could substantially improve the state-of-the-art performance on all the datasets.
BibTeX
@inproceedings{luo2023crowd,
  author = {Linbo Luo and Yuanjing Li and Haiyan Yin and Shangwei Xie and Ruimin Hu and Wentong Cai},
  title = {{Crowd-Level Abnormal Behavior Detection via Multi-Scale Motion Consistency Learning}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2023},
  doi = {10.1609/aaai.v37i7.26079},
}
Figure