Open-Space Vehicle Positioning Algorithm Based on Detection and Tracking in Video Sequences

Zhaoyu Zheng, Kai Zhang

TL;DR

This paper introduces an efficient and robust algorithm for open-space vehicle positioning, combining parallel detection and tracking with an improved DeepSORT model. It addresses the limitations of existing methods by reducing complexity and improving accuracy, even when vehicles or license plates are partially obscured, making it well-suited for on-street parking management.

What Problem Does This Paper Address?

Managing on-street parking lots efficiently is crucial, especially with the increasing number of vehicles. This requires accurately identifying vehicle positions and their corresponding license plate numbers for charging and management. Current approaches typically involve a sequential process: detecting vehicles, then their license plates, and finally recognizing the plate numbers. This multi-step process is computationally expensive and struggles when vehicles or their plates are obscured by other objects or at certain angles. The paper aims to overcome these challenges by developing a more streamlined and resilient system for open-space vehicle positioning.

Methods

The proposed algorithm is built around three core modules:

  • Parallel Detection and Matching (PDM): To tackle the high time and space complexity of traditional methods, this module performs parallel detection of both vehicles and license plates using a shared feature extraction network. This is followed by a geometric matching step to establish correspondences between detected vehicles and license plates. This parallel approach significantly reduces model parameters and inference time compared to sequential detection.
  • DeepSORT with Integrated Voting (DeepSORTv): This module enhances the well-known DeepSORT algorithm for tracking. The key innovation is the integration of "voting" for license plate number tracking. Instead of relying solely on the current frame, it leverages a historical license plate library and an integrated voting mechanism. This allows the system to infer license plate numbers even when they are temporarily obscured, improving robustness.
  • Cumulative State Detector (CSD): To accurately determine vehicle entry and exit events, a cumulative state detector is introduced. This method is designed to be more fault-tolerant than adjacent frame detectors, which are prone to errors due to data perturbations. The CSD tracks the vehicle's state over time, making more reliable judgments about entry and exit times.

Datasets

The authors created three custom datasets for evaluating their algorithm:

  • Vehicle Detection Dataset: Contains 4768 images with annotations for vehicles and license plates.
  • Vehicle Tracking Dataset: Consists of 16 videos (8 daytime, 8 nighttime) with frame-by-frame labeling to assess tracking performance.
  • Vehicle Entry-Exit Dataset: Comprises 156 videos (78 entry, 78 exit) labeled with license plate numbers and entry/exit times to evaluate the precision of these events.

These datasets were collected from on-street parking lots in Ningbo, China, using SNT-SC800 cameras.

Key Insights and Ideas

  • Parallelism is Key: Detecting vehicles and license plates simultaneously, rather than sequentially, dramatically cuts down on computational overhead and model size.
  • Geometric Matching for Correspondence: Using geometric relationships to link vehicles and license plates is an effective way to establish accurate associations without needing complex, separate matching algorithms.
  • Historical Data for Robustness: Leveraging past license plate information through integrated voting in DeepSORTv is a clever way to handle occlusions and improve tracking accuracy.
  • Cumulative State for Reliable Event Detection: Moving beyond single-frame analysis for entry/exit detection provides a more stable and accurate way to pinpoint these critical events.

Key Results

The proposed algorithm demonstrates significant improvements:

  • Detection: Achieves a Mean Average Precision ([email protected]) of 98.1% for all categories, with high precision and recall for both vehicles (95.6% precision, 97.2% recall) and license plates (98.2% precision, 96.2% recall).
  • Speed and Efficiency: The inference time is reduced to 20.4 ms per image, which is approximately 1/5th of the VLN method. The number of model parameters is also halved.
  • Tracking: Achieves a Multi-Object Tracking Accuracy (MOTA) of 92.1% and Multi-Object Tracking Precision (MOTP) of 89.4%, with a low number of ID switches (16 total).
  • License Plate Number Tracking (VLPA): The DeepSORTv module achieves a VLPA of 92.95%, a substantial improvement of 42.31 percentage points over standard DeepSORT.
  • Entry/Exit Time Precision: The Cumulative State Detector (CSD) shows higher precision compared to adjacent state detectors, especially for larger time error thresholds, with an accuracy improvement of 6.41 percentage points.

Qualitative results show high precision in bounding box predictions and remarkable accuracy in license plate number recognition, with vehicle IDs remaining consistent throughout tracking.

Limitations and Future Work

The primary limitation acknowledged by the authors is that the current work focuses on video detection and tracking from a single camera, which inherently limits the field of view.

Future work will concentrate on extending the system to handle multi-camera scenarios. This includes developing methods for multi-camera detection, cross-camera tracking, and enabling seamless collaboration between multiple cameras to cover larger areas.

Summary updated

Abstract

In on-street parking lots, it is very important to obtain the positions and license plate numbers of the vehicles for charging and management. Existing algorithms usually detect vehicles from an input image first, then localize their license plates, and finally recognize the license plate numbers. However, they are of high time and space complexity and cannot work if the vehicles or license plates are obscured. Therefore, this paper proposes an open-space vehicle positioning algorithm based on detection and tracking in video sequences. The work is as follows: (1) To reduce the time and space complexity, parallel detection of vehicles and license plates is carried out. Then, geometric matching is performed to accomplish the correspondences between them. (2) To track vehicles and license plates, this paper improves DeepSORT by combining with integrated voting based on the historical license plate library. (3) To accurately judge the vehicle behavior of entry or exit, a cumulative state detector is designed to increase the fault-tolerance of the proposed algorithm. The experimental results reveal that the proposed algorithm makes improvements in model parameters, inference speed, and tracking accuracy, demonstrating that it can be well applied to open-space vehicle positioning.
BibTeX
@inproceedings{zheng2022open,
  author = {Zhaoyu Zheng and Kai Zhang},
  title = {{Open-Space Vehicle Positioning Algorithm Based on Detection and Tracking in Video Sequences}},
  booktitle = {Sensors},
  year = {2022},
  url = {https://openreview.net/forum?id=WyWBv5xFiS},
}
Figure