Part-based Visual Tracking via Structural Support Correlation Filter

Zhangjian Ji, Kai Feng, Yuhua Qian

Abstract

Recently, part-based and support vector machines (SVM) based trackers have shown favorable performance. Nonetheless, the time-consuming online training and updating process limit their real-time applications. In order to better deal with the partial occlusion issue and improve their efficiency, we propose a novel part-based structural support correlation filter tracking method, which absorbs the strong discriminative ability from SVM and the excellent property of part-based tracking methods which is less sensitive to partial occlusion. Then, our proposed model can learn the support correlation filter of each part jointly by a star structure model, which preserves the spatial layout structure among parts and tolerates outliers of parts. In addition, to mitigate the issue of drift away from object further, we introduce inter-frame consistencies of local parts into our model. Finally, in our model, we accurately estimate the scale changes of object by the relative distance change among reliable parts. The extensive empirical evaluations on three benchmark datasets: OTB2015, TempleColor128 and VOT2015 demonstrate that the proposed method performs superiorly against several state-of-the-art trackers in terms of tracking accuracy, speed and robustness.

1. Introduction

Visual object tracking is a cornerstone of computer vision, yet it faces significant challenges such as partial occlusion, computational efficiency, and model drift. Traditional holistic trackers, which treat the target as a single entity, often fail when the target is partially hidden. Conversely, while SVM-based trackers offer high accuracy, their online optimization is typically too slow for real-time use. Furthermore, trackers often struggle with scale adaptation and drifting away from the target over time.

To address these issues, we introduce the Scale-adaptive Structural Support Kernel Correlation Filter (ScaleSSKCF). This method combines the discriminative power of SVMs with the efficiency of Correlation Filters, utilizing a part-based approach to enhance robustness against occlusions.

2. Methodology

The ScaleSSKCF framework is built upon four primary components: the Support Correlation Filter (SCF), a Star Structure Model, Temporal Consistency constraints, and Geometric Scale Estimation.

2.1 Support Correlation Filter (SCF)

The SCF merges the "max-margin" philosophy of SVMs with the computational speed of Correlation Filters. By operating in the Fourier domain, the model leverages the circulant property of matrices to reduce complexity from $O(n^4)$ to $O(n^2 \log n)$. The mathematical notations used in the formulation of these filters are detailed in the table below.

Table of mathematical notations and their explanations used in the ScaleSSKCF framework.

2.2 Star Structure and Temporal Constraints

Instead of treating parts independently, we employ a star structure model where each local part is connected to a central "dummy" part. This preserves the spatial layout of the object while allowing for individual part failures (e.g., due to occlusion). To further prevent model drift, inter-frame consistency constraints are introduced, ensuring that filter coefficients evolve smoothly across frames.

2.3 Scale Estimation

Scale changes are estimated by monitoring the relative distance changes between reliable parts. Reliability is determined using the Peak-to-Sidelobe Ratio (PSR) and appearance similarity. If parts move apart, the object is expanding; if they move closer, it is shrinking.

3. Experimental Evaluation

The proposed tracker was evaluated on three major benchmarks: OTB2015, TempleColor128, and VOT2015.

3.1 Performance Metrics

On the OTB2015 benchmark, ScaleSSKCF achieved a Mean Overlap Precision (OP) of 72.0%, outperforming state-of-the-art trackers like SRDCF and Staple. The tracker maintains a real-time speed of approximately 40 FPS on standard hardware.

3.2 Key Insights

  • Structural Importance: Ablation studies indicate that the star-structure constraint is the most critical factor for performance, preventing independent part drift.
  • Robustness to Occlusion: The part-based approach allows the tracker to maintain a lock on targets even during heavy occlusion, as seen in sequences like Human6.
  • Scale Accuracy: The relative-distance method for scale estimation proved more accurate than traditional pyramidal search methods, particularly during significant scale variations.

4. Conclusion and Future Work

While ScaleSSKCF demonstrates superior performance in many scenarios, it faces challenges with extremely fast motion where parts may move outside the search window. Future work will explore the integration of Deep Convolutional Features to further enhance accuracy, though balancing this with real-time performance remains a key consideration.

Summary updated

Abstract

Recently, part-based and support vector machines (SVM) based trackers have shown favorable performance. Nonetheless, the time-consuming online training and updating process limit their real-time applications. In order to better deal with the partial occlusion issue and improve their efficiency, we propose a novel part-based structural support correlation filter tracking method, which absorbs the strong discriminative ability from SVM and the excellent property of part-based tracking methods which is less sensitive to partial occlusion. Then, our proposed model can learn the support correlation filter of each part jointly by a star structure model, which preserves the spatial layout structure among parts and tolerates outliers of parts. In addition, to mitigate the issue of drift away from object further, we introduce inter-frame consistencies of local parts into our model. Finally, in our model, we accurately estimate the scale changes of object by the relative distance change among reliable parts. The extensive empirical evaluations on three benchmark datasets: OTB2015, TempleColor128 and VOT2015 demonstrate that the proposed method performs superiorly against several state-of-the-art trackers in terms of tracking accuracy, speed and robustness.
BibTeX
@article{ji2019part,
  author = {Zhangjian Ji and Kai Feng and Yuhua Qian},
  title = {{Part-based Visual Tracking via Structural Support Correlation Filter}},
  journal = {J. Vis. Commun. Image Represent.},
  year = {2019},
  url = {https://openreview.net/forum?id=Dqz8ZucHe1},
}
Figure