Introduction
Video anomaly detection (VAD) is a critical component of modern surveillance systems, yet it remains a challenging task due to the contextual nature of anomalies and the inherent noise in real-world footage. Traditional methods often struggle with unpredictable background fluctuations and varying object scales. This paper introduces ROADMAP (Robust unsupervised videO Anomaly Detection by Multi-path frAme Prediction), a framework designed to robustly predict normal video frames by ignoring random camera noise and focusing on semantic motion. By leveraging a multi-path architecture and a novel noise tolerance loss, ROADMAP achieves state-of-the-art performance on major VAD benchmarks.
The Challenge: Noise and Scale
In unsupervised video anomaly detection, models are typically trained on "normal" sequences to learn expected patterns. Deviations from these patterns during inference are flagged as anomalies. A common approach is frame prediction, where the model predicts the next frame and calculates the error relative to the ground truth.
However, two major hurdles persist:
- Pixel-Level Noise: Surveillance videos contain random fluctuations (sensor noise, lighting shifts) that are unpredictable. Standard models often waste capacity trying to reconstruct this meaningless noise rather than learning semantic motion.
- Varying Scales: Objects of interest (e.g., pedestrians vs. vehicles) appear at different scales depending on their distance from the camera, requiring a multi-resolution approach to capture dynamics effectively.
The ROADMAP Framework
To address these challenges, the authors propose a multi-path frame prediction network. Unlike standard U-Net architectures that rely on a single bottleneck, ROADMAP utilizes parallel prediction modules operating at different spatial resolutions.
As shown in the architecture diagram above, the network consists of three main components:
- Encoder: A CNN-based backbone that extracts hierarchical features.
- Predictor: Parallel paths containing Non-local blocks (for global spatial dependencies) and ConvGRU units (for local temporal dynamics). Shallow paths handle static backgrounds, while deeper paths focus on complex, multi-scale semantic motions.
- Decoder: Reconstructs the predicted frame $\hat{I}_t$ from the aggregated multi-path features.
Noise Tolerance Loss
A key innovation of ROADMAP is the Noise Tolerance Loss. Instead of relying solely on pixel-level Mean Squared Error (MSE), which forces the model to fit random noise, the authors introduce a perceptual loss. Both the predicted and ground-truth frames are passed through a pre-trained VGG16 network. The loss is calculated as the $L_1$ distance between intermediate feature maps.
As illustrated in the comparison above, raw pixel differences ($|I_t - I_{t-1}|$) are dominated by high-frequency background noise. In contrast, the feature-level differences ($|f_{nt}(I_t) - f_{nt}(I_{t-1})|$) effectively filter out this static, highlighting only the semantically meaningful motion of pedestrians. This ensures the model learns to predict actual content changes rather than sensor artifacts.
Experimental Evaluation
The model was evaluated on three standard benchmarks: CUHK Avenue, ShanghaiTech Campus, and UCSD Pedestrian (Ped1 & Ped2). Anomalies are detected by calculating the Peak Signal-to-Noise Ratio (PSNR) between the predicted and actual frames; a low PSNR indicates a high likelihood of an anomaly.
Key Results:
- State-of-the-Art Accuracy: ROADMAP achieved frame-level AUROC scores of 88.3% on Avenue, 76.6% on ShanghaiTech, and 96.3% on Ped2.
- Ablation Insights: The noise tolerance loss provided a significant boost on high-resolution datasets (e.g., +2.1% AUROC on ShanghaiTech). The multi-path design improved performance by 3.7% on the Avenue dataset compared to a single-path baseline.
- Efficiency: The system processes frames at approximately 10 FPS, making it suitable for near-real-time surveillance applications.
Conclusion and Limitations
ROADMAP demonstrates that filtering out irrelevant camera noise through perceptual loss and multi-scale temporal modeling significantly improves anomaly detection robustness. However, the authors note a limitation: the Noise Tolerance Loss relies on a VGG16 network pre-trained on ImageNet. This makes it less effective for low-resolution or grayscale videos, such as those in the UCSD Pedestrian datasets, where the loss weight had to be reduced to zero. Future research may focus on developing domain-specific noise-tolerant mechanisms for low-resolution environments.
Summary updated