Introduction
Understanding city-scale traffic flow is a cornerstone of modern intelligent transportation systems. However, automating vehicle counting by class and movement (e.g., distinguishing a left turn from a straight path) at complex intersections remains a significant challenge. Traditional supervised learning approaches are often hindered by the high cost of annotating vehicle trajectories for every unique camera angle and the difficulty of generalizing across diverse environmental conditions and camera distortions.
To address these issues, we present Zero-VIRUS (Zero-shot VehIcle Route Understanding System). This system eliminates the need for human-annotated trajectory data by modeling predefined traffic routes as continuous 2D fields, allowing for robust, zero-shot route identification.
System Architecture
The Zero-VIRUS pipeline operates in two primary stages: high-precision vehicle detection and tracking, followed by mathematical route modeling using spatial fields.
1. Detection and Trajectory Enhancement
The system utilizes a Mask R-CNN with a ResNeXt-101 backbone for robust object detection. To mitigate issues with overlapping detections—such as a car and truck being identified in the same spatial region—a Weighted Inter-class Non-Maximum Suppression (NMS) is employed, prioritizing "Car" detections to prevent double-counting.
A key innovation in the tracking phase is the shift from bounding box centers to segmentation-based localization. Standard bounding box centers often "float" in 2D space due to perspective, leading to inaccurate ground-plane trajectories. Zero-VIRUS instead extracts the vehicle's segmentation mask to identify the precise point where the tires contact the road. These paths are further refined using 1D-Gaussian smoothing and linear interpolation to maintain continuity during brief occlusions.
2. Route Modeling and Identification
Rather than training a classifier on specific movement patterns, Zero-VIRUS treats each possible route as a mathematical polyline converted into a 2D Field. This "zero-shot" approach evaluates vehicle tracklets against three core metrics:
- Proximity Field: Measures the spatial distance between the vehicle and the route, normalized by the vehicle's visual scale to account for perspective distortion.
- Completeness Field: Quantifies how much of a predefined route was traversed, filtering out "ghost" detections or partial movements.
- Stability Metric: Assesses whether the vehicle moves parallel to the route at a consistent distance, effectively filtering out erratic lane changes.
The final route classification is determined by a weighted aggregation of these three scores, providing a robust mechanism for movement categorization without scene-specific training.
Experimental Results
The system was evaluated on the AIC20_track dataset from the 2020 AI City Challenge. Zero-VIRUS achieved 3rd place overall with a score of 0.9292. Ablation studies confirmed the importance of the multi-metric approach; using all three fields (Proximity, Completeness, and Stability) yielded a significantly higher effectiveness score (0.9554) compared to using proximity alone (0.8903).
In terms of efficiency, the system processes approximately 9 frames per second on a single Nvidia RTX 2080Ti, demonstrating its potential for scalable, real-world deployment.
Conclusion
Zero-VIRUS demonstrates that high-accuracy vehicle route understanding can be achieved without expensive trajectory annotation. By leveraging segmentation-based tracking and 2D field modeling, the system provides a generalizable solution for complex urban intersections. Future work will focus on improving resilience to extreme weather conditions and further automating the definition of route polylines.
Summary updated