Geometry-Aware Traffic Flow Analysis by Detection and Tracking

Honghui Shi

TL;DR

This paper presents a method for estimating vehicle speeds from traffic camera footage by combining deep learning-based object detection and tracking with geometric camera calibration. Their approach achieved a top-tier performance in the 2018 Nvidia AI City Challenge, demonstrating the effectiveness of integrating geometric understanding with modern deep learning techniques for real-world traffic analysis.

What Problem Does This Paper Address?

Accurately estimating vehicle speeds from traffic camera videos is a crucial task for intelligent transportation systems, traffic management, and safety analysis. The challenge lies in the inherent difficulties of computer vision tasks: accurately detecting and tracking vehicles across frames, especially when they are partially occluded or move erratically, and then translating pixel-based movements into real-world speed measurements. Traditional methods often struggle with the dynamic nature of traffic and the varying perspectives of camera views. This paper tackles this by proposing a robust pipeline that leverages both powerful object detection and a geometrically informed speed estimation.

Methods

The authors propose a two-pronged approach:

  1. Detection and Tracking: They employ a state-of-the-art Mask R-CNN model (specifically, ResNet-101 Feature Pyramid Network) for detecting vehicles in frames. To overcome potential detection failures (vehicles missed in a frame), they use the Medianflow tracker to follow detected vehicles across subsequent frames. This ensures a more consistent trajectory for each vehicle. The center of the bounding boxes from both detection and tracking are used to represent vehicle positions.
  2. Geometry-aware Traffic Flow Analysis: This is the core innovation. They first estimate the vanishing point of the road lanes in each video. The vanishing point is a crucial geometric cue, representing the point where parallel lines in the real world appear to converge in the image. By understanding the relationship between pixel distances and real-world distances relative to this vanishing point, they can convert the apparent movement of a vehicle in pixels into actual displacement. This involves a mathematical formulation that uses the vanishing point, pixel distances, and angles to estimate real-world distances. The speed is then calculated using this estimated real-world distance and the video's frame rate. They also describe a method to calibrate the constant used in their distance estimation formula by using real-world road width measurements from sources like Google Maps.

Datasets

The paper primarily evaluates its method on the 27 videos provided for Track 1 of the Nvidia AI City Challenge 2018. They mention that they did not use the UADETRAC videos [20] due to uncertainties about the accuracy of their provided speed labels and missing camera information.

Key Insights and Ideas

  • Bridging the Gap: The paper highlights that combining robust deep learning detection/tracking with geometric understanding is more effective than relying on either alone. Geometric cues like the vanishing point provide a crucial link between the 2D image plane and the 3D real world.
  • Vanishing Point as a Geometric Anchor: The vanishing point of road lanes is a powerful invariant that allows for consistent geometric reasoning across frames, even if the camera experiences minor shakes.
  • Robustness to Detection Failures: Using a tracker like Medianflow significantly improves the continuity of vehicle trajectories, mitigating the impact of occasional missed detections by the primary object detector.
  • Simplicity and Effectiveness: The authors emphasize that their method, while conceptually simple, achieves high accuracy, suggesting that complex, end-to-end learned solutions aren't always necessary for achieving top performance.

Key Results

The proposed method achieved a root mean square error (RMSE) of 6.6674 in predicting vehicle speed, ranking them third place in the Nvidia AI City Challenge 2018. They also claim a perfect detection rate of target vehicles. The paper includes figures showing speed estimation results for various locations, visually demonstrating the method's performance.

Limitations and Future Work

  • Orthogonal Displacement: The authors acknowledge that their method is not capable of measuring horizontal speed if vehicles have horizontal displacements that are orthogonal to the lane direction within two frames.
  • Camera Information: They note that important camera parameters (like angle and height) were not provided for the challenge videos, which could further improve accuracy.
  • Manual Line Segment Selection: Currently, the identification of straight dash line segments for vanishing point estimation is done manually. Future work could involve developing a more automated way to detect these segments.
  • Further Improvements: The authors suggest that more advanced speed estimation could be pursued, especially after achieving high-performance detection and tracking. They also mention that incorporating more information about the camera's intrinsic and extrinsic parameters would be beneficial.

Summary updated

Abstract

In the second Nvidia AI City Challenge hosted in 2018, the traffic flow analysis challenge proposes an interest task that requires participants to predict the speed of vehicles on road from various traffic camera videos. We propose a simple yet effective method combing both learning based detection and geometric calibration based estimation. We use a learning based method to detect and track vehicles, and use a geometry based camera calibration method to calculate the speed of those vehicles. We achieve a perfect detection rate of target vehicles and a root mean square error (RMSE) of 6.6674 in predicting the vehicle speed, which rank us the third place in the competition.
BibTeX
@inproceedings{shi2018geometry,
  author = {Honghui Shi},
  title = {{Geometry-Aware Traffic Flow Analysis by Detection and Tracking}},
  booktitle = {Workshops @ CVPR},
  year = {2018},
  url = {https://openreview.net/forum?id=r14jR1fu-H},
}
Figure