Boundary Effect-Aware Visual Tracking for UAV with Online Enhanced Background Learning and Multi-Frame Consensus Verification

Changhong Fu, Ziyuan Huang, Yiming Li, Ran Duan, Peng Lu

Abstract

Visual tracking for Unmanned Aerial Vehicles (UAVs) faces unique challenges including fast motion, viewpoint changes, and frequent occlusions. Traditional Correlation Filter (CF) based trackers, while efficient, suffer from "boundary effects" caused by periodic assumptions in training. This paper presents the Boundary Effect-Aware Visual Tracker (BEVT), a framework that mitigates these issues through dense background learning and a multi-frame consensus verification system. By selectively updating the model based on a confidence-driven consensus score, BEVT significantly reduces tracking drift and improves robustness in aerial scenarios.

Introduction

Correlation Filters have become a staple in visual tracking due to their computational efficiency. However, their reliance on circular shifting to generate training samples introduces artificial boundary effects that degrade the discriminative power of the learned filters. In the context of UAV tracking, where the camera and target are often in high-speed motion, these inaccuracies lead to rapid tracking failure. Furthermore, standard trackers often update their models indiscriminately, making them susceptible to "pollution" from occluded or noisy frames.

The BEVT Framework

Online Enhanced Background Learning

To address the boundary effect, BEVT moves away from synthetic circular shifts. Instead, it densely extracts real background samples from a wider search area. To ensure the filter remains focused on the target despite the increased background information, a spatial penalization matrix is employed. This "bowl-shaped" penalty prioritizes the central region of the tracking window, allowing for a larger search radius without succumbing to distant background clutter.

Comparison of traditional circular shifting (top) vs. BEVT's real background learning (bottom). The bottom approach produces a much sharper response map peak, indicating higher discriminative power.

Deep Feature Integration and System Architecture

BEVT leverages deep features extracted from multiple layers (Conv3-4, Conv4-4, Conv5-4) of a pre-trained VGG-Net. This multi-level approach captures both fine-grained spatial details and high-level semantic context, which is crucial for handling viewpoint variations.

The complete BEVT framework architecture. It integrates deep feature extraction, spatial punishment, and a dual-filter system (initial and current) verified by a consensus loop.

Multi-Frame Consensus Verification

A core innovation of BEVT is its "Consensus Verification" mechanism. The tracker maintains an "ideal" response map and compares it to the current frame's response to calculate a consensus score ($C$).

  • High Confidence: If $C$ is high, the tracker aggressively updates the correlation filter to adapt to appearance changes.
  • Low Confidence: If $C$ is low (suggesting occlusion or blur), the tracker skips the update, preserving the integrity of the object model.

The resulting optimization problem is solved efficiently using the Alternating Direction Method of Multipliers (ADMM), enabling online operation.

Experimental Results

The tracker was rigorously evaluated on the UAV123_10fps dataset, comprising 100 challenging aerial sequences. BEVT was compared against 20 state-of-the-art trackers, including BACF, STRCF, and ECO variants.

  • Precision: BEVT achieved a precision score of 0.863, representing a 3.8% improvement over the runner-up (MCCT).
  • Success Rate: The Area Under Curve (AUC) for the success plot was 0.462, outperforming other methods.
  • Attribute-based Performance: BEVT showed superior robustness in specific challenging categories such as "Fast Motion" and "Viewpoint Change," validating its specialized design for UAV dynamics.

Conclusion and Future Work

The BEVT framework effectively addresses the limitations of traditional CF trackers in UAV environments by replacing synthetic shifts with real background learning and introducing a consensus-based update strategy. While achieving state-of-the-art accuracy, the current MATLAB implementation (0.653 FPS) is not yet suitable for real-time onboard deployment. Future work will focus on C++/GPU optimization and refining the spatial penalization matrix to further enhance performance in cluttered environments.

Summary updated

Abstract

Due to implicitly introduced periodic shifting of limited searching area, visual object tracking using correlation filters often has to confront undesired boundary effect. As boundary effect severely degrade the quality of object model, it has made it a challenging task for unmanned aerial vehicles (UAV) to perform robust and accurate object following. Traditional hand-crafted features are also not precise and robust enough to describe the object in the viewing point of UAV. In this work, a novel tracker with online enhanced background learning is specifically proposed to tackle boundary effects. Real background samples are densely extracted to learn as well as update correlation filters. Spatial penalization is introduced to offset the noise introduced by exceedingly more background information so that a more accurate appearance model can be established. Meanwhile, convolutional features are extracted to provide a more comprehensive representation of the object. In order to mitigate changes of objects' appearances, multi-frame technique is applied to learn an ideal response map and verify the generated one in each frame. Exhaustive experiments were conducted on 100 challenging UAV image sequences and the proposed tracker has achieved state-of-the-art performance.
BibTeX
@inproceedings{fu2019boundary,
  author = {Changhong Fu and Ziyuan Huang and Yiming Li and Ran Duan and Peng Lu},
  title = {{Boundary Effect-Aware Visual Tracking for UAV with Online Enhanced Background Learning and Multi-Frame Consensus Verification}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2019},
  url = {https://openreview.net/forum?id=Bqs3H0QmoHR},
}
Figure