Abstract
Researchers from the University of Science and Technology of China have introduced the Urban Surveillance Video Dataset (USVD), a comprehensive benchmark designed to advance multi-object tracking (MOT) and behavior analysis. Comprising over 3.7 million bounding boxes across 211,706 frames, USVD addresses the limitations of existing "clean" datasets by capturing the high congestion, frequent occlusions, and diverse viewpoints characteristic of real-world urban surveillance.
Introduction
While deep learning has significantly advanced computer vision, standard benchmarks like KITTI or the MOT Challenge often lack the complexity of 24/7 urban environments. In real-world city surveillance, cameras frequently encounter extreme congestion where dozens of pedestrians, cyclists, and vehicles overlap. Current algorithms often struggle when transitioning from sparse laboratory data to these dense, noisy environments. USVD fills this gap by providing a large-scale, realistic dataset designed to stress-test the next generation of urban AI.
Methods and Data Curation
The authors curated USVD by collecting nearly 4,000 hours of raw video from 54 surveillance cameras across diverse outdoor scenarios, including crossroads, school gates, and parks. From this pool, 16 representative scenes were selected for manual annotation.
The annotation process followed strict consistency rules. Seven distinct classes were identified: pedestrian, riding (person + vehicle), car, van, bus, tricycle, and truck. Notably, a person on a bicycle or scooter is treated as a single "riding" unit, reflecting how urban planners and security systems typically view traffic flow. To maintain accuracy in crowded scenes, targets smaller than 60 pixels were handled with specific protocols.
The USVD Dataset
USVD represents a significant leap in scale and density compared to existing benchmarks. It contains 32 sequences and approximately 7,100 unique trajectories. With over 3.7 million bounding boxes, it is more than ten times the size of the popular MOT Challenge 2017. The dataset's density is a key feature, averaging 18-20 objects per frame. This high density ensures that models must account for frequent "mutual occlusion," where objects block one another, rather than just simple motion patterns.
Key Insights: Scale Variation and Congestion
A primary challenge identified in USVD is extreme scale variation. As objects approach or move away from the camera, their size in pixels changes dramatically. The authors found that over 32% of targets undergo "huge" scale changes, which is a frequent cause of tracking failure in standard models.
Experimental Results
The authors established baselines using standard detection and tracking models:
- Detection: The Single Shot MultiBox Detector (SSD) achieved a mean Average Precision (mAP) of 0.6656, slightly outperforming Faster R-CNN and demonstrating robustness for real-time urban applications.
- Tracking: The TC_ODAL algorithm reached the highest Multiple Object Tracking Accuracy (MOTA) at 41.05%. Interestingly, the simpler IOU tracker performed nearly as well (40.94%), suggesting that spatial overlap is a powerful cue in high-density scenes.
- Anomaly Detection: Using the Histogram of Optical Flow (HMOF) method, the researchers achieved an Area Under Curve (AUC) of 0.883 for identifying anomalous behaviors like running in restricted zones.
Conclusion and Future Work
USVD provides a rigorous new standard for urban surveillance AI. While currently restricted by licensing, its scale and complexity offer a unique testing ground for scale-invariant and occlusion-robust algorithms. Future work includes annotating "group" dynamics (people walking together) and expanding the dataset to include varied weather conditions such as rain, snow, and nighttime footage to further challenge AI robustness.
Summary updated