Introduction
Detecting anomalies in surveillance video—such as unauthorized vehicles or unusual pedestrian behavior—remains a significant challenge in computer vision. Traditional detection methods often struggle with two primary issues: high sensitivity to environmental noise (shadows, lighting changes) and fragmented detections where a single anomalous object is perceived as disconnected parts. This paper proposes the MultiLevel Anomaly Detector (MLAD), a framework designed to mitigate these issues by integrating low-level visual features with high-level semantic representations.
Methodology: The MLAD Framework
The MLAD system processes video through a multilevel approach, leveraging both appearance and motion data across various layers of abstraction.
The framework operates in three main phases:
- Representation Learning: Denoising Autoencoders (DAEs) are trained on raw video frames (intensity) and optical flow maps (motion). By extracting activations from different layers of these DAEs, the system captures a range of features from simple edges (low-level) to complex object shapes and spatial relationships (high-level).
- Cross-Domain Generation with cGANs: For every representation level, the system trains a pair of Conditional Generative Adversarial Networks (cGANs). One GAN predicts the motion map given the appearance frame, while the other predicts the appearance frame given the motion map. In "normal" scenes, the relationship between appearance and motion is predictable; if the GANs fail to accurately reconstruct one from the other, it indicates an anomaly.
- Multilevel Consolidation: Finally, the system generates error maps at each level and merges them. High-level detections, which are more robust but less precise, help validate and complete low-level detections, which are precise but often noisy.
Key Insights and Results
A core strength of MLAD is its ability to leverage "objectness." While low-level features might only detect high-contrast edges, higher-level layers encode semantic identities, allowing the system to identify an anomaly as a single continuous entity rather than a collection of moving pixels.
The authors found that DAEs trained specifically on surveillance scenes outperformed general-purpose features from models like AlexNet. As shown in the comparison above, MLAD provides a more coherent and localized detection map. This is attributed to the fact that surveillance anomalies are often small and local, whereas models like AlexNet are optimized for large, global object classification.
Performance Benchmarks
The model was evaluated on the UCSD Ped 1, Ped 2, and CUHK Avenue datasets. Notably, the authors provided a re-annotated ground truth for UCSD Ped 1 to correct previous labeling errors involving missing anomalies and mislabeled occlusions. MLAD achieved state-of-the-art results, specifically improving the pixel-level Equal Error Rate (EER) by:
- UCSD Ped 1: 11.35% improvement.
- UCSD Ped 2: 12.32% improvement.
- Avenue: 4.31% improvement.
Conclusion and Future Work
MLAD effectively reduces false positives and fragmented detections by combining multilevel representations with cross-modal validation. While the computational cost of running multiple GANs is a consideration, the authors suggest that a simplified version using only low and high-level layers (MLAD_0+3) offers an optimal balance of speed and accuracy. Future research directions include integrating temporal consistency across longer sequences of frames to further refine detection stability and reduce noise.
Summary updated