Abstract
Speedy abnormal event detection meets the growing demand to process an enormous number of surveillance videos. Based on inherent redundancy of video structures, we propose an efficient sparse combination learning framework. It achieves decent performance in the detection phase without compromising result quality. The short running time is guaranteed because the new method effectively turns the original complicated problem to one in which only a few costless small-scale least square optimization steps are involved. Our method reaches high detection rates on benchmark datasets at a speed of 140-150 frames per second on average when computing on an ordinary desktop PC using MATLAB.
Introduction
In modern surveillance, abnormal events—such as running, loitering, or unauthorized vehicle access—are rare but critical to detect. Traditional computer vision approaches often utilize sparse representation, where a "dictionary" of normal behaviors is built, and new frames are flagged if they cannot be reconstructed efficiently from this dictionary. However, the computational cost of solving optimization problems for every frame has historically limited these methods to near-offline speeds. This paper introduces a breakthrough framework that achieves real-time performance (150 FPS) in MATLAB by shifting the computational burden from the testing phase to the training phase through sparse combination learning.
Methodology
Spatial-Temporal Feature Extraction
The system processes video by dividing it into a multi-scale spatial-temporal pyramid. It extracts "cubes" consisting of 5 consecutive frames across three different scales to capture both fine-grained and coarse appearance and motion patterns.
From these cubes, 3D gradient features are extracted and reduced to 100 dimensions using Principal Component Analysis (PCA). This representation ensures that the system captures essential motion dynamics while remaining computationally light.
Sparse Combination Learning
The core innovation lies in replacing the expensive $L_0$ or $L_1$ optimization typically used in sparse coding with a set of pre-learned basis combinations. During the training phase, the algorithm identifies $K$ groups of basis vectors that represent common "normal" patterns.
During the testing phase, instead of searching the entire dictionary, the system performs a simple least-squares check against these pre-learned combinations $S_1, \dots, S_K$. This matrix projection is extremely fast. Furthermore, an "early exit" strategy is employed: since most surveillance footage is normal, the system often finds a matching combination quickly (checking only ~32.5% of combinations on average) and proceeds to the next frame.
Experimental Evaluation
The framework was evaluated on three major benchmarks:
- Avenue Dataset: A new dataset containing 15 sequences with 14 unusual events like running and throwing objects.
- UCSD Ped1: A standard benchmark for pedestrian abnormality (e.g., bikes on a sidewalk).
- Subway Dataset: Long-duration (2-hour) videos of subway entrances and exits, focusing on events like walking the wrong way or jumping turnstiles.
Performance and Accuracy
The method achieved a processing speed of 140–150 FPS on a standard 3.4GHz CPU, which is approximately 400x faster than previous sparsity-based approaches. Despite this speed, accuracy remained high; for instance, it achieved an Area Under the Curve (AUC) of 91.8% on the UCSD Ped1 dataset. On the Subway dataset, processing time per frame was reduced to a mere 0.006 seconds. This proves that the mathematical formulation itself is inherently efficient, rather than relying on low-level hardware hacks.
Conclusion
By leveraging the inherent redundancy in surveillance footage, the proposed sparse combination learning framework demonstrates that high-speed, high-accuracy abnormal event detection is possible on consumer-grade hardware. While currently optimized for static cameras, this approach sets a new standard for efficient real-time video analytics, proving that MATLAB can be a viable environment for high-performance computer vision tasks.
Summary updated