AirCamRTM: Enhancing Vehicle Detection for Efficient Aerial Camera-Based Road Traffic Monitoring

Rafael Makrigiorgis, Nicolas Hadjittoouli, Christos Kyrkou, Theocharis TheocharidesWACV 2022

TL;DR

This paper introduces AirCamRTM, a novel framework that integrates road segmentation with vehicle detection from aerial imagery to improve the efficiency and accuracy of road traffic monitoring. By focusing detection efforts only on road areas, AirCamRTM achieves a 2x speedup and an 18% accuracy improvement over traditional methods, making real-time traffic analysis from UAVs more feasible.

What Problem Does This Paper Address?

Efficient road traffic monitoring is crucial for managing urban congestion and optimizing transportation systems. While Unmanned Aerial Vehicles (UAVs) offer a flexible, infrastructure-free solution for this, real-time monitoring from aerial imagery presents significant challenges. These include handling large image sizes, distinguishing relevant vehicles from non-target objects (like parked cars or objects on sidewalks), and reducing the computational burden for real-time processing. Existing approaches often focus on either vehicle detection or road segmentation independently, failing to leverage the synergy between these tasks for enhanced performance.

Methods

The AirCamRTM framework is a composite visual processing pipeline built upon deep learning. It consists of three main stages:

  1. Road Segmentation: The paper explores various encoder-decoder architectures (e.g., UNet, FPN, PAN, DeepLabV3) combined with different encoders (EfficientNet, MobileNetV2, ResNet18) to predict a road mask. This mask is designed to highlight relevant road areas, thereby guiding the subsequent vehicle detection.
  2. Vehicle Detection: The authors investigate single-shot detectors from the YOLO family (YOLOv3, YOLOv4, and their "tiny" versions) for their speed and accuracy trade-offs. They specifically select Tiny YOLOv4@3layers for its competitive performance on a GPU-equipped laptop, achieving 25 FPS.
  3. Post-Processing and Integration:
    • Mask Correction: The predicted road mask is refined using detected bounding boxes to ensure accurate coverage of the road area.
  4. Temporal Association and Consistency: To track vehicles over time and estimate trajectories/velocities, the Hungarian algorithm and Kalman Filter are employed. This involves matching bounding boxes across frames using IoU scores and maintaining vehicle state estimates.
  5. Velocity and Direction Calculation: Once vehicles are tracked, their velocity and direction are computed using pixel-to-meter conversion and frame-to-frame coordinate differences.

The core novelty lies in the synergistic integration of road segmentation to pre-filter regions of interest for vehicle detection, significantly reducing computational load and improving accuracy.

Datasets

The paper utilizes several datasets for training and evaluation:

  • Vehicle Detection:
    • UAVDT Dataset: Re-annotated into four classes ('Cars', 'Busses', 'Trucks', 'Motorbikes').
  • Custom UAV Footage: Collected from real UAV flights in Cyprus, annotated from scratch into the same four classes.
  • Road Segmentation:
    • OpenStreetMap (OSM) Dataset: Used for training road segmentation models.
  • Composite Evaluation:
    • Aerial KITTI Dataset: Used for joint evaluation of the integrated framework. Ground truth labels were modified to include only on-road vehicles.

Key Insights and Ideas

  • Synergy of Segmentation and Detection: The most significant insight is that explicitly segmenting the road area before performing vehicle detection dramatically improves efficiency and accuracy. This is because it filters out irrelevant detections (e.g., parked cars, vehicles off-road) that would otherwise consume computational resources and potentially lead to false positives.
  • Focus on Relevant Vehicles: The framework's ability to focus only on vehicles on the main road is crucial for real-time traffic monitoring, especially in complex urban environments. This directly addresses the challenge of computational complexity when dealing with numerous objects.
  • Trade-offs in Segmentation Architectures: The paper highlights that while UNet-based models are popular, they may not perform as well for aerial road segmentation compared to FPN or PAN decoders, especially when combined with efficient encoders like ResNet.
  • Tiny YOLO for Real-time UAV Applications: The selection of Tiny YOLOv4@3layers demonstrates that efficient, yet accurate, vehicle detection is achievable on resource-constrained platforms like a laptop ground station for UAV operations.

Key Results

  • Performance Improvement: The AirCamRTM framework achieved approximately a 2x speedup and an 18% accuracy improvement compared to road-agnostic approaches.
  • Vehicle Detection Accuracy: On the Aerial KITTI test dataset, using the segmentation mask led to higher accuracy, F1-score, and mAP compared to using full images. For example, accuracy increased from 0.41 to 0.59 without and with segmentation, respectively.
  • Frame Rate: The integrated framework achieved an average of 20 FPS with the road mask applied, a significant improvement over approximately 8 FPS without the mask.
  • Segmentation Performance: The segmentation architecture achieved 77.7% mIOU and 96.3% mean accuracy on the Aerial KITTI dataset.
  • Impact of Vehicle Count: The number of vehicles significantly impacts tracking performance, underscoring the need for efficient filtering provided by the road segmentation.

Limitations and Future Work

  • Detector Reliance: The framework's performance is dependent on the detector's ability to identify vehicles. If the detector fails, the pipeline cannot function.
  • Occlusions: Long-period occlusions (e.g., by bridges, tall buildings) can hinder detector functionality. The authors suggest mitigating this with more consistency rules and drone positioning adjustments.
  • Road Prediction Sensitivity: Road prediction can be affected by drone altitude, position, shadows, and road network complexity. Iterative mask correction helps, but challenges remain.
  • Future Work: The authors plan to further integrate detection and segmentation networks to avoid redundant processing and improve generalization. They also aim to expand datasets with data from different times and conditions for both segmentation and detection.

Summary updated

BibTeX
@inproceedings{makrigiorgis2022aircamrtm,
  author = {Rafael Makrigiorgis and Nicolas Hadjittoouli and Christos Kyrkou and Theocharis Theocharides},
  title = {{AirCamRTM: Enhancing Vehicle Detection for Efficient Aerial Camera-Based Road Traffic Monitoring}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2022},
}
Figure