Abstract
Automated detection of traffic anomalies, such as accidents and stalled vehicles, is critical for urban management but remains challenging due to chaotic scenes, varying weather, and a lack of labeled data. This paper introduces a modular, unsupervised framework developed by researchers from Baidu and Sun Yat-sen University. The system utilizes multi-granularity tracking—combining box-level and pixel-level analysis—alongside a "backward background modeling" technique to accurately identify anomalies. The proposed method achieved a 98.5% F1-score, securing first place in the NVIDIA AI CITY 2020 Challenge.
Introduction
Detecting traffic accidents automatically is a significant objective for intelligent transportation systems and emergency response. However, computer vision models often struggle with the diversity of traffic environments and the scarcity of labeled accident data. Traditional unsupervised systems frequently fail to distinguish between true anomalies and normal stationary behavior, such as vehicles in parking lots or auxiliary roads. This research addresses these challenges by proposing a robust framework that isolates stationary vehicles on main roads while filtering out environmental noise through modular components.
Methodology
The framework is structured into four modular stages to ensure robustness and precision:
- Detection & Background Modeling: A Faster R-CNN with a SENet-152 backbone is employed for vehicle detection, while a Gaussian Mixture Model (MOG2) models the background. By subtracting the background, the system isolates stationary vehicles, which serve as primary indicators of potential anomalies.
- Hypothetical Abnormal Mask Extraction: To mitigate false positives in non-road areas (e.g., parking lots), a mask is generated using motion-based data and DeepSORT trajectories, focusing the analysis solely on the main roadway.
- Multi-Granularity Tracking: This core component employs two parallel branches to capture complementary information:
- Box-level Tracking: Links detections into temporal "tubes" to track individual objects across frames.
- Pixel-level Tracking: Analyzes spatial-temporal matrices to detect subtle pixel changes that indicate the onset of a vehicle stall.
- Fusion and Backtracking Optimization: Results from both branches are fused. To address detection latency, a "backward background modeling" technique is applied, analyzing the video in reverse from the point of detection to pinpoint the exact start time of the anomaly.
Experimental Evaluation
The framework was evaluated using the NVIDIA AI CITY 2020 Challenge (Track 4) dataset, which consists of 200 videos (100 training, 100 test) recorded at 30 fps with a resolution of 800 × 410. The dataset presents significant challenges, including varying lighting and adverse weather conditions such as snow.
The system demonstrated state-of-the-art performance, achieving the following metrics:
- F1-Score: 0.9855
- RMSE (Root Mean Square Error): 4.8737 seconds
- S4 Score: 0.9695 (the primary competition metric combining F1 and RMSE)
Discussion and Conclusion
A key insight of this work is the "stationary heuristic," which assumes that vehicles stopped on main roads beyond a typical traffic cycle are likely anomalies. The integration of backward modeling significantly reduces the delay in anomaly detection, while the modular mask approach effectively prunes false positives.
Despite its success, the system is primarily tuned for stopped vehicles and may require further adaptation for dynamic anomalies, such as wrong-way driving at high speeds. Future work could also address the computational complexity of the multi-component architecture to enhance real-time deployment capabilities in large-scale urban environments.
Summary updated