Vehicle Counting Based on Vehicle Detection and Tracking from Aerial Videos

Xuezhi Xiang, Mingliang Zhai, Ning Lv, Abdulmotaleb El-SaddikSensors 2018 · OpenReview

TL;DR

This paper introduces a robust framework for vehicle counting from UAV-captured aerial videos, effectively handling both static and dynamic backgrounds. By combining an adaptive foreground detector, an online-learning tracker, and a multi-threading management module, the system achieves high accuracy in complex traffic scenarios.

What Problem Does This Paper Address?

Traditional traffic monitoring sensors (e.g., loop detectors, cameras) often have limitations such as installation costs, road damage, and restricted coverage. While surveillance cameras are common, they struggle with issues like occlusion and limited views. Unmanned Aerial Vehicles (UAVs) offer a flexible and cost-effective solution for monitoring large areas from a bird's-eye perspective. However, analyzing aerial video for vehicle counting presents unique challenges: complex backgrounds, varying vehicle scales and shapes, and the need to track multiple vehicles simultaneously, especially when the UAV itself is in motion. This paper aims to address these challenges by proposing a comprehensive system for accurate vehicle counting from UAV-based aerial videos, even in the presence of moving backgrounds.

Methods

The proposed framework consists of four main components:

  • Vehicle Detection:
    • Static Background: Employs the ViBe (Visual Background Extractor) algorithm, which uses pixel-level background subtraction and an adaptive model that updates continuously to handle minor scene changes.
  • Moving Background: Utilizes image registration techniques (specifically, SURF features and RANSAC) to estimate camera motion and warp frames to a reference coordinate system, allowing for vehicle detection in a stable frame.
  • Multi-Vehicle Tracking: Leverages an online-learning tracker based on ridge regression. This tracker updates its training samples dynamically, enabling it to adapt to changes in vehicle scale and shape within the video sequence.
  • Multi-Vehicle Management Module: Addresses the limitation of single-object trackers by designing a module that manages multiple tracked vehicles. This module uses multi-threading to assign tracking tasks to parallel blocks, significantly improving computational efficiency and enabling robust analysis of individual vehicle status. It also handles the identification of new vehicles entering the scene and reinitializes trackers when necessary.
  • Vehicle Counting: The final step uses the information from the multi-vehicle management module (vehicle IDs and directions) to count vehicles within a defined area, eliminating the need for multiple virtual test lines.

Datasets

The authors used a dataset of real freeway scene videos recorded by a UAV at resolutions of 960x540. The data was divided into two groups based on UAV height (50m and 100m) and two flight modes (static hovering and linear horizontal flight). A total of 16 test videos were used for evaluation. Ground truth for vehicle counting was obtained by manually examining videos frame by frame.

Key Insights and Ideas

  • UAVs as a superior platform: The paper strongly advocates for UAVs as a more flexible and comprehensive solution for traffic monitoring compared to traditional sensors.
  • Handling moving backgrounds is crucial: The proposed image registration method effectively compensates for UAV motion, a significant challenge in aerial video analysis.
  • Online learning for adaptability: The use of an online-learning tracker is key to handling the varying appearance of vehicles due to scale and shape changes in aerial views.
  • Multi-threading for scalability: The multi-object management module's reliance on multi-threading is a critical design choice for efficiently processing multiple vehicles and reducing computational load.
  • Unified framework: The integration of detection, tracking, and management into a single framework is essential for reliable vehicle counting.

Key Results

The proposed method achieved high accuracy on real-world highway scenes:

  • Static Background: Over 90% accuracy.
  • Moving Background: Over 85% accuracy.

The paper also demonstrated the effectiveness of its multi-threading approach, showing significant speedup in processing compared to sequential methods when tracking multiple objects. Parameter tuning was shown to be important for optimal performance, with specific settings yielding the highest accuracy.

Limitations and Future Work

  • Limitations:
    • The paper notes that for the height of 100m, the average error rate is slightly higher than at 50m, attributing this to some small objects being misclassified as background by the detector.
  • The accuracy of moving background scenarios is lower than static backgrounds, indicating that errors in camera motion estimation can impact detection results.
  • Future Work: Not explicitly detailed in the provided text, but the current framework lays a strong foundation for further improvements in areas like handling extreme occlusion, adverse weather conditions, and real-time deployment on resource-constrained UAV platforms.

Summary updated

Abstract

Vehicle counting from an unmanned aerial vehicle (UAV) is becoming a popular research topic in traffic monitoring. Camera mounted on UAV can be regarded as a visual sensor for collecting aerial videos. Compared with traditional sensors, the UAV can be flexibly deployed to the areas that need to be monitored and can provide a larger perspective. In this paper, a novel framework for vehicle counting based on aerial videos is proposed. In our framework, the moving-object detector can handle the following two situations: static background and moving background. For static background, a pixel-level video foreground detector is given to detect vehicles, which can update background model continuously. For moving background, image-registration is employed to estimate the camera motion, which allows the vehicles to be detected in a reference coordinate system. In addition, to overcome the change of scale and shape of vehicle in images, we employ an online-learning tracker which can update the samples used for training. Finally, we design a multi-object management module which can efficiently analyze and validate the status of the tracked vehicles with multi-threading technique. Our method was tested on aerial videos of real highway scenes that contain fixed-background and moving-background. The experimental results show that the proposed method can achieve more than 90% and 85% accuracy of vehicle counting in fixed-background videos and moving-background videos respectively.
BibTeX
@inproceedings{xiang2018vehicle,
  author = {Xuezhi Xiang and Mingliang Zhai and Ning Lv and Abdulmotaleb El-Saddik},
  title = {{Vehicle Counting Based on Vehicle Detection and Tracking from Aerial Videos}},
  booktitle = {Sensors},
  year = {2018},
  url = {https://openreview.net/forum?id=4GAj6ekIk8},
}
Figure