A Novel Vehicle Tracking ID Switches Algorithm for Driving Recording Sensors

Yun Zhao, Xiang Zhou, Xing Xu, Zeyu Jiang, Fupeng Cheng, Jiahui Tang, Yuan Shen2020 · DOI

TL;DR

This paper introduces a novel algorithm to reduce "ID switches" in vehicle tracking from driving recorder sensors. By combining enhanced appearance features with existing tracking methods, the algorithm significantly improves the continuity of vehicle identification, a crucial aspect for advanced driver-assistance systems (ADAS).

What Problem Does This Paper Address?

Driving data recorders (DDRs) are increasingly common in vehicles, providing valuable data for understanding road conditions and driver behavior. A key task for DDRs is real-time vehicle tracking, which involves identifying and maintaining consistent IDs for vehicles across consecutive video frames. A major challenge in this process is "ID switching," where a vehicle is incorrectly assigned a new ID after a temporary occlusion (e.g., being hidden behind another vehicle or an obstacle). This can lead to errors in analyzing traffic flow, accident reconstruction, and the overall reliability of ADAS. Existing methods, while good at detection, often struggle with maintaining consistent IDs during these occlusions, especially in complex urban environments with frequent interactions between vehicles.

Methods

The proposed method builds upon the Deep SORT algorithm, a popular framework for object tracking. The core innovation lies in an improved approach to handling ID switches. The authors integrate appearance features more effectively into the tracking process. Specifically, they use a modified Deep SORT algorithm that incorporates:

  • YOLOv3 for Detection: The system utilizes YOLOv3 for initial vehicle detection due to its speed and accuracy.
  • Kalman Filter for Prediction: A Kalman filter is employed to predict the future positions of detected vehicles.
  • Fusion of Distance Metrics: The algorithm calculates multiple distance metrics to match predicted vehicle states with detected ones. These include:
  • Mahalanobis Distance: Measures the distance between predicted and detected positions, considering the covariance of the measurements.
  • Cosine Distance with Appearance Metrics: This is a key enhancement. It incorporates appearance features extracted by a deep neural network (specifically, a network with convolutional layers and residual blocks) into the cosine similarity calculation. This helps in re-identifying vehicles even after occlusion by considering their visual characteristics.
  • Euclidean Distance: Used for initial matching and to manage track deletion based on time intervals.
  • Cascading Matching Strategy: A strategy is employed to prioritize matching closer targets, which is beneficial when dealing with occlusions.
  • Track Management: Vehicles are categorized into tentative, confirmed, and deleted states to manage their tracking status.

The primary goal is to reduce ID switches by making the association between predicted and detected bounding boxes more robust, especially when appearance information is fused with positional data.

Datasets

The paper utilizes two primary datasets:

  • MS COCO (Common Objects in Context): A large-scale dataset for object detection, containing 80 categories and 1.5 million object instances. It was used for training and evaluating the detection model.
  • BDD100K (Berkeley DeepDrive): A diverse dataset specifically designed for autonomous driving, containing numerous videos collected by the Nexar application. It includes various weather conditions (sunny, rainy, foggy, snowy) and times of day (day, night). The dataset was used for training and evaluating the tracking algorithm, with a focus on the "car" category.

Key Insights and Ideas

  • Appearance Matters for Continuity: The paper strongly emphasizes that relying solely on positional information (like in the original SORT algorithm) is insufficient for robust tracking, especially during occlusions. Incorporating appearance features is crucial for maintaining consistent vehicle IDs.
  • Fusion is Key: Simply using Mahalanobis distance or cosine distance alone is not enough. The fusion of positional (Mahalanobis) and appearance-based (cosine with appearance metrics) distances leads to a more accurate and stable matching process.
  • BDD100K's Value for Robustness: Training on the BDD100K dataset, with its diverse conditions (weather, lighting), significantly improves the model's ability to handle challenging scenarios like night driving and rain, leading to better detection and tracking performance compared to models trained solely on COCO.
  • ID Switches are a Major Hurdle: The paper highlights that ID switches are a persistent and significant problem in vehicle tracking, particularly in real-world driving scenarios with frequent occlusions.

Key Results

The proposed Modified Deep SORT (MDS) algorithm demonstrated significant improvements in reducing ID switches:

  • BDD100K Trained Model: Achieved a 29.95% reduction in ID switches compared to the original Deep SORT.
  • COCO Trained Model: Achieved a 32.16% reduction in ID switches compared to the original Deep SORT.

The paper also notes that the BDD100K trained model performed better at night and in rainy conditions, and was more effective at detecting small and distant targets. The tracking speed for pre-shot videos was 4.5 fps, and for real-time camera processing, it reached 6.5 fps.

Limitations and Future Work

  • Dataset Scope: While BDD100K offers diversity, the authors suggest expanding the dataset to further train the model and mitigate errors like misclassifying truck wheels as cars.
  • Appearance Feature Improvement: Although appearance features are incorporated, further improvements in this area could lead to even fewer ID switches.
  • Recognition Speed: While the algorithm is real-time, further optimization of recognition speed is mentioned as a potential area for improvement.
  • Higher Resolution Cameras: The authors suggest using higher-resolution cameras for DDRs to capture better image quality, which would aid recognition.
  • Multi-Camera Tracking: Future work could involve tracking vehicles across different video cameras and developing better methods for matching appearance features across multiple viewpoints.

Summary updated

BibTeX
@misc{zhao2020novel,
  author = {Yun Zhao and Xiang Zhou and Xing Xu and Zeyu Jiang and Fupeng Cheng and Jiahui Tang and Yuan Shen},
  title = {{A Novel Vehicle Tracking ID Switches Algorithm for Driving Recording Sensors}},
  year = {2020},
  doi = {10.3390/s20133638},
}
Figure