Abstract
In crowded scenes, detection and localization of abnormal behaviors is challenging because high-density crowds make object segmentation and tracking extremely difficult. This paper presents a method that associates the optical flows of multiple frames to capture short-term trajectories and introduces a histogram-based shape descriptor, referred to as shape contexts, to describe these trajectories. Furthermore, we propose a K-NN similarity-based statistical model to detect anomalies over time and space. This unsupervised one-class learning algorithm requires no clustering or prior assumptions. By retrieving K-NN samples from the training set and constructing a Gaussian model based on their intra-class similarities, we calculate the joint probability of a test sample's similarity. Abnormal events are detected when this probability falls below predefined thresholds. This scheme adapts to the entire scene, as the probability is not affected by motion distortions arising from perspective distortion. Experiments on real-world surveillance videos demonstrate that the proposed method reliably detects and locates abnormal events, outperforming state-of-the-art approaches.
Introduction: The Challenge of Crowded Scenes
Detecting anomalies in dense crowds is a significant hurdle in computer vision. Traditional methods typically rely on segmenting and tracking individual objects. However, in a dense crowd, individuals constantly overlap and obscure one another, causing standard trackers to fail.
Beyond density, unsupervised models often struggle with "perspective distortion." In a typical surveillance shot, objects far from the camera appear smaller and move fewer pixels per second than those close to the camera. Using a fixed distance threshold to define "normal" behavior often leads to false positives, where a normal person walking near the camera is flagged as an anomaly simply because their pixel-velocity is higher than the training data from the background. This research seeks a method robust to these challenges without requiring labeled "abnormal" data for training.
Methodology
The proposed pipeline consists of three primary stages designed for efficiency and robustness:
- Short-term Trajectory Capture: Rather than attempting long-term tracking, the system uses optical flow to track "particles" over a window of 10 frames. This captures the immediate "shape" of motion without the complexity and failure points of long-term object tracking.
- Shape Context Descriptors: These short-term trajectories are converted into histogram-based descriptors called "shape contexts." This summarizes the distribution of motion within a local patch in a computationally efficient manner.
- K-NN Similarity-based Statistical Learning: This core innovation involves finding the $K$ nearest neighbors in the normal training data for any new test patch. Instead of measuring raw distance, the system builds a Gaussian model based on the intra-class similarity of those $K$ neighbors. It then calculates the joint probability that the test patch fits into that similarity distribution. If the probability is too low, the patch is flagged as an anomaly.
This "one-class" learning approach is advantageous because it only requires "normal" video for training and bypasses the need for pre-defined clusters, which are often difficult to maintain in dynamic real-world environments.
Experimental Evaluation
The researchers evaluated the model on two standard benchmarks: the UCSDped1 dataset, which features significant perspective distortion and anomalies like bikers and carts on pedestrian paths, and the Subway Dataset, which includes anomalies like loitering and wrong-way walking.
Quantitative Results
The proposed method outperformed several state-of-the-art (SOTA) approaches, including Social Force Models (SFM) and Mixture of Dynamic Textures (MDT). On the UCSDped1 dataset, the method achieved an Equal Error Rate (EER) of 25.6% at the pixel level, a marked improvement over the 37.3% achieved by Gaussian Process Regression (GPR).
Localization and Efficiency
Qualitatively, the model proved highly effective at localizing anomalies. Rather than flagging an entire frame, the system draws specific grids around the anomalous objects. The system is also designed for real-time application, processing frames in approximately 175 milliseconds—significantly faster than competing spatio-temporal composition methods.
Conclusion
By modeling the distribution of similarities rather than raw feature distances, this method naturally scales with perspective. If a distant, small biker appears, the model compares it to other distant, small normal pedestrians, maintaining local consistency. While optical flow can be noisy in extremely dense crowds, the current results show a robust ability to identify and locate "weird" behaviors in unsupervised settings. Future work may explore multi-scale analysis to improve object coverage and investigate how similarity-based logic handles contextual anomalies where the location itself defines the abnormality of an action.
Summary updated