Vision-Based Real-Time Aerial Object Localization and Tracking for UAV Sensing System

Yuanwei Wu, Yao Sui, Guanghui Wang

Abstract

This paper introduces FOLT (Fast Object Localization Tracking), a novel framework designed for Unmanned Aerial Vehicles (UAVs) to autonomously detect and track aerial obstacles in real-time. By integrating a saliency-based detector with a dynamic Kalman filter, the system eliminates the need for manual initialization while achieving state-of-the-art accuracy at a processing speed of 141 frames per second (FPS).

Introduction

For UAVs to operate safely in shared or complex airspace, a robust "Sense and Avoid" (SAA) system is essential. Vision-based solutions are preferred for their low cost and weight, yet they typically struggle with three challenges: the requirement for manual target initialization, high computational demands of automated detection (e.g., deep learning or optical flow), and environmental dynamics like rapid scale changes or lighting variations. FOLT addresses these hurdles by providing a training-free, computationally efficient pipeline for fully autonomous tracking.

Methodology

The FOLT framework treats object localization and tracking as a recursive, two-stage process governed by a Kalman Filter.

  • Saliency-Based Detection: Rather than identifying specific object classes, the system identifies "salient" regions that contrast with the background. It utilizes a background connectivity cue, assuming image edges are likely background, and calculates the minimum barrier distance to highlight distinct foreground objects.
  • Dynamic Kalman Modeling: The system maintains a state vector for position, velocity, and size. The Kalman filter predicts the object's future location, allowing the detector to focus only on a localized "search region" rather than the full frame, significantly reducing latency.
  • Refinement and Correction: Once a salient region is identified within the search area, adaptive thresholding and morphological operations generate a precise bounding box. This measurement is then used to correct the Kalman filter’s state for the next cycle.
The FOLT framework integrates object localization and post-processing within a recursive Kalman Filter loop.

Experimental Results

The system was evaluated on 15 challenging image sequences (including Aircraft and Skyjumping) featuring scale variations, rotations, and background interference.

Quantitative Performance

FOLT was compared against seven state-of-the-art trackers, including KCF, DSST, and SAMF. Key findings include:

  • Speed: FOLT reached an average of 141.3 FPS, making it roughly 10x faster than SAMF and comparable to the high-speed KCF (which requires manual initialization).
  • Accuracy: In One Pass Evaluation (OPE), FOLT achieved a Precision of 0.83, significantly outperforming the next best tracker (CCT at 0.66).
  • Robustness: The system demonstrated a 13% improvement in success rate over competitors, proving highly effective at handling rapid rotations and size changes.
Precision and Success plots for One Pass Evaluation (OPE) and Temporal Robustness Evaluation (TRE) comparing FOLT against state-of-the-art trackers.

Discussion and Insights

A key innovation of FOLT is its use of temporal context. While traditional saliency detectors treat frames in isolation, FOLT uses the object's movement history to narrow the search space, reducing both false positives and computational load. Furthermore, because the system is training-free and unsupervised, it can track novel objects it has never encountered before, unlike deep learning models that require extensive labeled datasets.

However, the system faces limitations during severe occlusion. If a target is obscured by clouds or smoke for an extended period, the tracker may lose the target. While FOLT includes re-localization capabilities, performance inevitably dips during total invisibility.

Conclusion

FOLT provides a high-speed, autonomous solution for UAV obstacle tracking. By combining efficient saliency detection with recursive Kalman filtering, it achieves a balance of speed and precision suitable for onboard deployment. Future work will focus on enhancing re-detection logic to handle prolonged occlusions and integrating the system into closed-loop collision avoidance controllers.

Summary updated

BibTeX
@article{wu2017vision,
  author = {Yuanwei Wu and Yao Sui and Guanghui Wang},
  title = {{Vision-Based Real-Time Aerial Object Localization and Tracking for UAV Sensing System}},
  journal = {IEEE Access},
  year = {2017},
  doi = {10.1109/access.2017.2764419},
}
Figure