TL;DR
This paper introduces "Keyfilter-Aware Object Tracking" (KAOT), a novel approach for real-time object tracking on Unmanned Aerial Vehicles (UAVs). By incorporating "keyfilters" derived from periodically selected keyframes, KAOT effectively mitigates filter corruption and reduces redundant context learning, leading to improved accuracy and efficiency for challenging UAV tracking scenarios.
What Problem Does This Paper Address?
Tracking objects from the perspective of a UAV presents unique challenges. The high maneuverability of UAVs, combined with complex backgrounds, frequent appearance variations (occlusion, deformation, illumination changes), and limited onboard computational resources, makes robust and efficient real-time tracking difficult. Traditional discriminative correlation filter (DCF) methods, while fast, suffer from a "boundary effect" due to artificial sample generation and are prone to filter corruption from appearance changes. Existing solutions to these problems, like enlarging the search area or frame-by-frame context learning, often introduce background distractions or are computationally too expensive for real-time UAV applications.
Methods
The core innovation of KAOT is the introduction of keyfilters. Inspired by keyframe techniques in Simultaneous Localization and Mapping (SLAM), keyfilters are generated from periodically selected keyframes. These keyfilters then "restrict" the learning of the main tracking filters. This restriction serves two main purposes:
- Context Awareness Transmission: Keyfilters ensure that context learned at keyframes is propagated to all subsequent filters, providing a more consistent understanding of the environment.
- Filter Corruption Repression: By periodically updating the keyfilter and using it to constrain the main filter, KAOT mitigates the corruption that can arise from continuous, unconstrained learning in the face of appearance variations.
The paper also mentions the use of intermittent context learning to reduce redundancy, as context often changes much slower than the frame rate of a UAV camera. The optimization of the tracking model is handled using the Alternating Direction Method of Multipliers (ADMM). For feature extraction, KAOT combines hand-crafted features (HOG, CN) with deep features (conv3 layer from VGG-M).
Datasets
The paper evaluates KAOT on two challenging UAV datasets:
- DTB70: This dataset is noted for being recorded with more frequent and drastic displacements.
- UAV123@10fps: This dataset captures sequences at 10 frames per second.
In total, 193 image sequences were used for evaluation. No new benchmarks were introduced; the evaluation was performed using existing protocols for these datasets.
Key Insights and Ideas
- Keyfilters as a "memory" for context: The idea of using keyframes to periodically "reset" or guide the learning of filters is a clever way to balance learning new information with maintaining stability. It's like giving the tracker a periodic "snapshot" of what's important.
- Reducing redundancy in context learning: The observation that context changes much slower than camera frame rates is crucial. KAOT's intermittent approach avoids computationally expensive, frame-by-frame context updates when they aren't necessary.
- Bridging the gap between DCF and deep learning: KAOT effectively combines the speed advantages of DCF with the representational power of deep features, aiming for both high accuracy and real-time performance.
- Addressing specific UAV tracking challenges: The paper directly tackles issues like boundary effects and filter corruption, which are particularly problematic in the dynamic and often challenging visual environments encountered by UAVs.
Key Results
- Overall Performance: KAOT demonstrates competitive performance on both DTB70 and UAV123@10fps datasets, often ranking among the top trackers in precision. On DTB70, it achieved an average precision of 0.757.
- Speed: KAOT achieves a speed of approximately 15 FPS on a GPU, making it suitable for real-time UAV applications. It is noted to be significantly faster (27%) than ECO while achieving higher precision.
- Attribute-Based Performance: KAOT ranks first in precision on five out of six challenging attributes tested (fast camera motion, background clutter, in-plane rotation, deformation, and similar objects around). It shows particular strength in handling background clutter and deformation due to its keyfilter mechanism.
- Comparison with Non-Real-Time Trackers: KAOT outperforms five non-real-time trackers in both precision and speed on the tested benchmarks.
Limitations and Future Work
- Keyframe Selection: The current keyframe selection mechanism is periodic. The authors acknowledge that this could lead to issues if a keyframe itself is unreliable. They suggest that a more adaptive strategy for choosing keyframes could further improve robustness.
- Re-detection and Rotation: While KAOT handles appearance changes well, it is limited when the object disappears for extended periods or in situations of significant rotation. The authors propose incorporating re-detection and rotation-aware modules to address these limitations.
- Optimization: The current implementation is in MATLAB and not fully optimized, suggesting that further speed improvements are possible.
Summary updated