Visual Tracking Based on Multi-Feature and Fast Scale Adaptive Kernelized Correlation Filter

Xianyou Zeng, Long Xu, Yigang Cen, Ruizhen Zhao, Shaohai Hu, Guohui XiaoIEEE Access 2019 · OpenReview

Abstract

Tracking methods based on a correlation filter have attracted much attention because of their high efficiency and strong robustness. However, a tracker based on a single feature is obviously not sufficient to adapt to the complex appearance changes of the target. Besides, rapid and exact scale estimation is still a challenging problem in the field of visual tracking. In this paper, we introduce an independent scale filter for the estimation of the scale of an object and merge two complementary features to further boost the performance of the tracker. At the same time, a dimension reduction strategy is adopted to decrease the computational load. Finally, a dynamic learning rate-based model update mechanism is inserted to effectively alleviate model degradation problem by suppressing the influence of noisy appearance changes. The extensive experiments were conducted on the object tracking benchmark (OTB) dataset and Temple color 128 dataset. The quantitative and qualitative results exhibit that compared with other popular trackers, the tracker proposed in this paper acquires favorable results in tracking accuracy, efficiency, and robustness. On the OTB-2015 benchmark dataset, it obtains precision scores of 0.773, 0.782, and 0.714 and success scores of 0.585, 0.606, and 0.534 in the three indexes of OPE, TRE, and SRE. On the Temple color 128 dataset, it acquires precision scores of 0.641, 0.681, and 0.606 and success scores of 0.478, 0.515, and 0.445 in the three indexes of OPE, TRE, and SRE, surpassing many well-known tracking methods. In terms of tracking efficiency, it runs at a speed of 42.3 frames/s on a single CPU, making it suitable for real-time applications.

Introduction

Visual object tracking is a fundamental task in computer vision, essential for applications ranging from surveillance to autonomous driving. While Kernelized Correlation Filter (KCF) based trackers have gained popularity due to their efficiency, they often struggle with real-world "nuisances" such as illumination changes, occlusions, and deformations. This paper introduces the MFFSKCF tracker, an evolution designed to overcome the limitations of standard KCF frameworks through multi-feature fusion and adaptive scaling.

Addressing the Gaps in Visual Tracking

The authors identify three critical weaknesses in traditional correlation filter trackers:

  1. Feature Insufficiency: Relying on a single feature (e.g., HOG) often fails when target appearance changes drastically, such as a colorful object moving against a similarly textured background.
  2. Fixed Scale: Basic filters assume a constant target size, leading to drift as objects move relative to the camera.
  3. Rigid Model Updates: Fixed-rate updates can cause the model to "learn" occlusions or noise, leading to gradual model degradation and eventual tracking failure.
Scale estimation is a significant challenge; standard trackers often fail to adapt to size changes, as illustrated by the varying bounding boxes.

The MFFSKCF Methodology

The proposed Multi-Feature and Fast Scale Adaptive Kernelized Correlation Filter (MFFSKCF) integrates four key innovations to address these challenges:

Multi-Feature Fusion and Dimension Reduction

The tracker extracts both Histogram of Oriented Gradients (HOG) for structural information and Color Attributes (CN) for robustness against motion blur. These features are fused to create a more robust response map. To maintain real-time performance, a PCA-based dimension reduction strategy compresses high-dimensional features before processing, ensuring the Fast Fourier Transforms (FFTs) remain efficient.

The process of extracting multiple feature maps and vectorizing them for correlation filter processing.

Fast Scale Adaptation

Instead of an exhaustive search, MFFSKCF employs an independent 1D correlation filter to estimate scale changes across a scale pyramid. This allows the tracker to adjust the bounding box size dynamically and accurately without significant computational overhead.

Dynamic Learning Rate

To prevent model degradation, the system uses a confidence-based update mechanism. By calculating the Peak-to-Sidelobe Ratio (PSR) of the response map, the tracker assesses tracking confidence. If the PSR is low—indicating potential occlusion or noise—the learning rate is automatically reduced to protect the model from being corrupted by bad data.

Experimental Evaluation and Results

The MFFSKCF tracker was rigorously tested on the OTB-2013, OTB-2015, and Temple Color 128 benchmarks.

  • Accuracy and Robustness: On the OTB-2015 benchmark, it achieved a precision score of 0.773 and a success score of 0.585, outperforming the baseline KCF by approximately 11%. On the challenging Temple Color 128 dataset, it maintained a high precision of 0.641, surpassing well-known trackers like SAMF and DSST.
  • Real-Time Efficiency: Despite its increased complexity, the tracker operates at 42.3 FPS on a single CPU, making it highly suitable for practical, real-time applications.

Conclusion

By combining multi-feature fusion, independent scale filtering, and a dynamic update mechanism, the MFFSKCF tracker provides a robust and efficient solution for visual tracking. Future work will explore adaptive feature weighting and the integration of deep learning features to further enhance tracking precision in complex environments.

Summary updated

Abstract

Tracking methods based on a correlation filter have attracted much attention because of their high efficiency and strong robustness. However, a tracker based on a single feature is obviously not sufficient to adapt to the complex appearance changes of the target. Besides, rapid and exact scale estimation is still a challenging problem in the field of visual tracking. In this paper, we introduce an independent scale filter for the estimation of the scale of an object and merge two complementary features to further boost the performance of the tracker. At the same time, a dimension reduction strategy is adopted to decrease the computational load. Finally, a dynamic learning rate-based model update mechanism is inserted to effectively alleviate model degradation problem by suppressing the influence of noisy appearance changes. The extensive experiments were conducted on the object tracking benchmark (OTB) dataset and Temple color 128 dataset. The quantitative and qualitative results exhibit that compared with other popular trackers, the tracker proposed in this paper acquires favorable results in tracking accuracy, efficiency, and robustness. On the OTB-2015 benchmark dataset, it obtains precision scores of 0.773, 0.782, and 0.714 and success scores of 0.585, 0.606, and 0.534 in the three indexes of OPE, TRE, and SRE. On the Temple color 128 dataset, it acquires precision scores of 0.641, 0.681, and 0.606 and success scores of 0.478, 0.515, and 0.445 in the three indexes of OPE, TRE, and SRE, surpassing many well-known tracking methods. In terms of tracking efficiency, it runs at a speed of 42.3 frames/s on a single CPU, making it suitable for real-time applications.
BibTeX
@inproceedings{zeng2019visual,
  author = {Xianyou Zeng and Long Xu and Yigang Cen and Ruizhen Zhao and Shaohai Hu and Guohui Xiao},
  title = {{Visual Tracking Based on Multi-Feature and Fast Scale Adaptive Kernelized Correlation Filter}},
  booktitle = {IEEE Access},
  year = {2019},
  url = {https://openreview.net/forum?id=W-dwepU3Bo},
}
Figure