Abstract
Abnormal event detection in video is a challenging vision problem. Most existing approaches formulate abnormal event detection as an outlier detection task, due to the scarcity of anomalous data during training. Because of the lack of prior information regarding abnormal events, these methods are not fully-equipped to differentiate between normal and abnormal events. In this work, we formalize abnormal event detection as a one-versus-rest binary classification problem. Our contribution is two-fold. First, we introduce an unsupervised feature learning framework based on object-centric convolutional auto-encoders to encode both motion and appearance information. Second, we propose a supervised classification approach based on clustering the training samples into normality clusters. A one-versus-rest abnormal event classifier is then employed to separate each normality cluster from the rest. For the purpose of training the classifier, the other clusters act as dummy anomalies. During inference, an object is labeled as abnormal if the highest classification score assigned by the one-versus-rest classifiers is negative. Comprehensive experiments are performed on four benchmarks: Avenue, ShanghaiTech, UCSD and UMN. Our approach provides superior results on all four data sets. On the large-scale ShanghaiTech data set, our method provides an absolute gain of 8.4% in terms of frame-level AUC compared to the state-of-the-art method.
Introduction
Detecting abnormal events in video—such as a vehicle on a pedestrian path or a person running in a restricted area—is a critical task for automated surveillance. Traditionally, this is treated as an outlier detection problem: models are trained on purely "normal" data and flag anything that deviates from the learned distribution. However, this approach often suffers from context blindness, where background noise (like swaying trees) triggers false alarms, and a lack of discriminative power, as the model never learns what an anomaly actually looks like.
Ionescu et al. propose a shift from unsupervised outlier detection to a discriminative classification framework. By focusing on specific objects and leveraging a novel "dummy anomaly" training strategy, they significantly improve the robustness and accuracy of anomaly detection.
Methodology: Object-Centric Learning and Dummy Anomalies
The proposed framework consists of two primary stages: feature extraction via convolutional auto-encoders (CAEs) and classification using a one-versus-rest strategy.
Object-Centric Feature Learning
To avoid background distractions, the system uses a pre-trained Single Shot Detector (SSD) to isolate objects. For each detected object, the model extracts two types of information:
- Appearance: A grayscale crop of the object.
- Motion: Temporal gradients calculated between the current frame ($t$) and adjacent frames ($t-3, t+3$).
These inputs are processed by three separate CAEs (one for appearance and two for motion). The auto-encoders are trained to reconstruct the input, and the resulting latent representations are concatenated into a single feature vector that captures both how an object looks and how it moves.
The "Dummy Anomaly" Strategy
The core innovation lies in how the model is trained without explicit anomaly labels. The researchers use K-means clustering to group normal training samples into $k$ distinct clusters (e.g., "walking," "standing," "cycling").
Instead of training a single "normality" model, they train $k$ separate one-versus-rest Support Vector Machine (SVM) classifiers. For a given cluster (e.g., "walking"), all other normal clusters (e.g., "cycling") are treated as dummy anomalies. This forces the classifier to learn tight decision boundaries around specific normal behaviors. During inference, if an object's features result in negative scores from all $k$ classifiers, it is deemed an anomaly.
Experimental Results
The method was evaluated on four benchmark datasets: Avenue, ShanghaiTech, UCSD Ped2, and UMN.
Performance Metrics
The framework achieved state-of-the-art (SOTA) performance across all benchmarks. Most notably, on the large-scale ShanghaiTech dataset, it reached a frame-level Area Under the Curve (AUC) of 84.9%, representing an 8.4% absolute improvement over previous methods. On the Avenue dataset, it achieved 90.4% AUC.
Qualitative Analysis
As shown in the qualitative results, the model successfully identifies various anomalies such as running, wrong-way movement, and throwing objects. The object-centric approach allows for precise localization of these events. However, some false positives occur, often due to detector errors or unusual but benign poses that the model hasn't seen in its training clusters.
Conclusion
By transforming anomaly detection into a discriminative classification task through the use of dummy anomalies, this work provides a powerful new direction for video surveillance. The object-centric focus ensures that the model remains attentive to the actual actors in a scene, while the CAE-based features provide a rich representation of behavior. While the system's performance is currently tied to the accuracy of the underlying object detector, its significant gains on complex datasets like ShanghaiTech demonstrate the effectiveness of the "dummy anomaly" paradigm. Future work exploring object tracking could further enhance temporal consistency and detection accuracy.
Summary updated