Robust and Fast Vehicle Turn-counts at Intersections via an Integrated Solution from Detection, Tracking and Trajectory Modeling

Zhihui Wang, Bing Bai, Yujun Xie, Tengfei Xing, Bineng Zhong, Qinqin Zhou, Yiping Meng, Bin Xu, Zhichao Song, Pengfei Xu, Runbo Hu, Hua Chai

Accurately counting vehicle turns at busy intersections is a cornerstone of modern urban planning and traffic signal optimization. However, automating this task using single-camera feeds remains a significant challenge due to occlusions, adverse weather, and perspective distortions. This paper, presented at CVPR 2020 workshops, introduces an integrated pipeline that combines deep learning-based detection with traditional background modeling and a sophisticated trajectory matching schema to achieve high accuracy and real-time performance.

The Challenge of Intersection Monitoring

Standard computer vision approaches often struggle in real-world traffic scenarios. Several factors contribute to this difficulty:

  • Occlusions: Vehicles frequently obscure one another, especially in heavy traffic.
  • Environmental Factors: Heavy rain, glare, and shifting shadows can confuse even advanced neural network detectors.
  • Perspective Distortion: High-angle "floating" camera views make it difficult to distinguish specific lanes.
  • Fragmented Tracks: Temporary loss of a vehicle by a tracker leads to "broken" trajectories, causing double-counting or missed counts.
Challenging environmental conditions including rain and glare across different intersection perspectives.

An Integrated Pipeline for Turn-Counting

The authors propose a sequential pipeline consisting of five integrated modules designed to move beyond "one-size-fits-all" models.

The proposed integrated pipeline for vehicle turn-counting, spanning from perception to final results.

1. Hybrid Vehicle Perception

To improve recall in extreme weather, the system combines a high-performance deep learning detector (NAS-FPN) with GMM-like background modeling. While deep learning is the industry standard, the authors found that traditional background subtraction provides a necessary "safety net," capturing moving objects that a neural network might miss due to low visibility or unusual lighting.

2. Robust Tracking and Trajectory Modeling

The system utilizes DeepSort for multi-object tracking, enhanced by a custom post-processing step to smooth trajectories and filter out static noise like parked cars. A key innovation is the creation of a lane-level gallery. By aggregating a large number of "clean" trajectories, the system builds a template of valid movements for a specific intersection.

3. Spatial Trajectory Matching

Most trackers rely on timing, but traffic lights make vehicle duration unpredictable. Instead, the authors segment paths based on spatial distance. This allows the system to match a noisy, real-time path to the most likely template in the lane-level gallery, effectively "snapping" fragmented tracks to valid movements.

4. Fragment Stitching

To address broken tracks, the system calculates a vehicle's "survival time." If two short trajectory fragments have matching survival times and non-conflicting velocities, they are merged and treated as a single vehicle.

Performance and Results

The method was evaluated on the AICity 2020 Track1 dataset, which includes 20 real-world scenes across 31 video sequences featuring various weather conditions and complex traffic.

Tested on a Tesla P40 GPU, the system achieved:

  • S1-Score: 92.60% (combining detection and counting accuracy).
  • Effectiveness: 91.40%.
  • Speed: 13 FPS, making it viable for near-real-time monitoring.

Conclusion and Future Work

The integrated approach demonstrates that combining modern deep learning with traditional spatial geometry and background modeling creates a robust solution for traffic analysis. While highly effective, the authors note that identity switches still occur during complex, long-duration occlusions (e.g., a large truck blocking a car). Future work may involve incorporating visual appearance features to better re-identify vehicles after they emerge from obstructions.

Summary updated

Abstract

In this paper, we address the problem of vehicle turn- counts by class at multiple intersections, which is greatly challenged by inaccurate detection and tracking results caused by heavy weather, occlusion, illumination variations, background clutter, etc. Therefore, the complexity of the problem calls for an integrated solution that robustly ex- tracts as much visual information as possible and efficiently combines it through sequential feedback cycles. We pro- pose such an algorithm, which effectively combines detection, background modeling, tracking, trajectory modeling and matching in a sequential manner. Firstly, to improve detection performances, we design a GMM like background modeling method to detect moving objects. Then, the pro- posed GMM like background modeling method is combined with an effective yet efficiency deep learning based detector to achieve high-quality vehicle detection. Based on the detection results, a simple yet effective multi-object tracking method is proposed to generate each vehicle's movement trajectory. Conditioned on each vehicle's trajectory, we then propose a trajectory modeling and matching schema which leverages the direction and speed of a local vehicle's trajectory to improve the robustness and accuracy of vehicle turn-counts. Our method is validated on the AICity Track1 dataset A, and has achieved 91.40% in effectiveness, 95.4% in efficiency, and 92.60% S1-score, respectively. The experimental results show that our method is not only effective and efficient, but also can achieve robust counting performance in real-world scenes.
BibTeX
@inproceedings{wang2020robust,
  author = {Zhihui Wang and Bing Bai and Yujun Xie and Tengfei Xing and Bineng Zhong and Qinqin Zhou and Yiping Meng and Bin Xu and Zhichao Song and Pengfei Xu and Runbo Hu and Hua Chai},
  title = {{Robust and Fast Vehicle Turn-counts at Intersections via an Integrated Solution from Detection, Tracking and Trajectory Modeling}},
  booktitle = {Workshops @ CVPR},
  year = {2020},
  doi = {10.1109/cvprw50498.2020.00313},
}
Figure