Online Learning-Based Hybrid Tracking Method for Unmanned Aerial Vehicles

Sohee Son, Injae Lee, Jihun Cha, Haechul ChoiSensors 2023 · DOI

TL;DR

This paper introduces a novel hybrid tracking method for Unmanned Aerial Vehicles (UAVs) that combines object detection and tracking with an online learning mechanism. This approach effectively addresses challenges like changing appearances, diverse sizes, and out-of-view scenarios, leading to more robust and accurate UAV tracking.

What Problem Does This Paper Address?

Tracking unmanned aerial vehicles (UAVs) in real-world scenarios is a significant challenge. UAVs exhibit rapid and unpredictable motion, vary greatly in size, and can change their appearance due to lighting conditions or orientation. Furthermore, they frequently disappear from view (out-of-view) or become partially obscured (occlusion). Existing tracking methods often struggle with these issues, leading to drift or complete loss of the target. This paper aims to overcome these limitations by developing a system that can robustly track UAVs even under these difficult conditions.

Methods

The proposed method is a hybrid approach that integrates three key components: a detector, a tracker, and an integrator.

  • Detector: A deep learning-based object detector (specifically YOLOv4 is mentioned as an example) is used to identify potential UAVs in the scene.
  • Tracker: A conventional tracking algorithm (MedianFlow is used as an example) is employed to follow the detected object across frames.
  • Integrator: This is the core novelty of the proposed method. It acts as a decision-maker, combining the outputs of the detector and tracker. Crucially, it uses an online learning-based AdaBoost classifier to update the target's features in real-time. This online adaptation allows the system to adjust to changes in the UAV's appearance, deformation, or scale without requiring extensive offline training on every possible scenario. The integrator predicts the confidence of both the detector and tracker's outputs and uses a threshold to decide which output to trust, or if the target is absent. This mechanism helps to recover from tracking failures and correct for detection errors.

Datasets

The authors utilized a combination of datasets for their experiments:

  • Custom UAV Dataset: A unique dataset was constructed by recording videos of UAVs in diverse outdoor environments. This dataset includes sequences with challenging attributes like occlusion (OC), out-of-view (OV), background clutter (BC), camera motion (CM), low resolution (LR), scale variation (SV), and fast motion (FM).
  • Drone-vs-Bird Dataset: A public dataset was also used to enhance the generalizability of the proposed method.
  • UAV123 and UAVL Datasets: These are established public datasets commonly used for aerial tracking evaluation, further demonstrating the method's performance across different benchmarks.

Key Insights and Ideas

  • Online Learning is Crucial for Dynamic Environments: The paper highlights that static, offline training is insufficient for robust UAV tracking. The ability to adapt and learn online is essential to handle the constantly changing nature of UAV appearances and environmental conditions.
  • Detection as a Recovery Mechanism: Integrating a detector with a tracker provides a powerful way to recover from tracking failures. When the tracker loses the target, the detector can re-localize it, preventing permanent loss.
  • Hybrid Approach Outperforms Individual Components: The experimental results consistently show that the combined detection-tracking approach, especially with the online learning integrator, outperforms using either a detector or a tracker alone.
  • Balancing Accuracy and Speed: The proposed method offers a trade-off between accuracy and processing speed, allowing for effective real-time applications by adjusting the complexity of the online classifier.

Key Results

  • Tracking Performance: The proposed method achieved a success score of 0.747 on a custom dataset, outperforming other methods like LC (0.719) and YOLOv4 (0.618). It also demonstrated strong performance on the UAV123 dataset, achieving the best success scores in 8 out of 12 scenarios.
  • Detection Performance: The method achieved a high precision of 0.836 and the best recall value, indicating its ability to maintain steady tracking without losing the target. The F-measure was reported as 0.879.
  • Robustness: The method showed significant robustness in handling challenging scenarios such as out-of-view, occlusion, and scale variation, often recovering tracks where other methods failed.
  • Speed: While not the fastest, the proposed method achieved a processing speed of 21.90 FPS, which is considered effective for real-time applications requiring over 20 FPS.

Limitations and Future Work

  • High Density of Dynamic and Tiny Objects: In scenes with a very high density of small, fast-moving objects, the detection and tracking might still struggle, potentially contaminating the online classifier.
  • Processing Time: The integration of detection and tracking mechanisms inherently increases processing time. The authors suggest that implementing multi-threading could help mitigate this.
  • Classifier Contamination: If the detector or tracker provides incorrect estimations, it can negatively impact the online-learned classifier, potentially leading to drift.
  • Future Work: The authors suggest that the framework's performance is dependent on the integrated detection and tracking algorithms. Improving these components could further enhance the proposed method. They also note that the framework can be applied to other computer vision tasks.

Summary updated

Abstract

Tracking unmanned aerial vehicles (UAVs) in outdoor scenes poses significant challenges due to their dynamic motion, diverse sizes, and changes in appearance. This paper proposes an efficient hybrid tracking method for UAVs, comprising a detector, tracker, and integrator. The integrator combines detection and tracking, and updates the target's features online while tracking, thereby addressing the aforementioned challenges. The online update mechanism ensures robust tracking by handling object deformation, diverse types of UAVs, and changes in background. We conducted experiments on custom and public UAV datasets to train the deep learning-based detector and evaluate the tracking methods, including the commonly used UAV123 and UAVL datasets, to demonstrate generalizability. The experimental results show the effectiveness and robustness of our proposed method under challenging conditions, such as out-of-view and low-resolution scenarios, and demonstrate its performance in UAV detection tasks.
BibTeX
@inproceedings{son2023online,
  author = {Sohee Son and Injae Lee and Jihun Cha and Haechul Choi},
  title = {{Online Learning-Based Hybrid Tracking Method for Unmanned Aerial Vehicles}},
  booktitle = {Sensors},
  year = {2023},
  doi = {10.3390/s23063270},
}
Figure