Scene-Aware Adaptive Updating for Visual Tracking via Correlation Filters

Fan Li, Sirou Zhang, Xiaoya Qiao

TL;DR

This paper introduces an "Adaptive Kernel Correlation Filter" (AKCF) that dynamically adjusts its update strategy based on the detected scene conditions (deformation, occlusion, etc.). This allows for more robust and accurate visual object tracking, especially in challenging scenarios where traditional methods often fail.

What Problem Does This Paper Address?

Visual object tracking is crucial for many applications, but real-world scenarios are fraught with challenges like scale variation, deformation, occlusion, and fast motion. Traditional correlation filter-based trackers, while fast, often struggle with these dynamic conditions. They tend to update their models uniformly, which can lead to incorporating occluded or deformed parts into the model, causing tracking drift or failure when the target reappears or its appearance changes drastically. This paper aims to improve the robustness and accuracy of correlation filter trackers by making their adaptation mechanisms "scene-aware."

Methods

The proposed AKCF algorithm builds upon the Kernel Correlation Filter (KCF) framework and introduces two main novelties:

  • Low Complexity Scale Estimation: Instead of exhaustive scale search, the AKCF estimates the target scale by considering five candidate scales (two zoomed in, two zoomed out, and the original). Each scale's response is weighted based on its prior probability, allowing for efficient and accurate scale estimation without significantly sacrificing speed.
  • Scene-Aware Adaptive Updating Mechanism: This is the core contribution. The algorithm first classifies the current video scene into one of four categories: deformation, no occlusion/deformation, partial occlusion, or long-time large-area occlusion. This classification is based on three feature parameters: similarity (between current and previous frames), holistic response (of the KCF tracker), and block response (analyzing responses of segmented target blocks). Based on the classified scene, a specific adaptive updating strategy is employed for the target feature template and the learning rate. For instance, in deformation scenes, a high learning rate is used to quickly adapt, while in long-time large-area occlusion, updating is suspended to prevent learning occluded parts.

Datasets

The authors evaluate their algorithm on the CVPR2013 benchmark (OTB50), which contains 50 standard test videos. They specifically analyze performance on 19 deformation videos and 29 occlusion videos.

Key Insights and Ideas

  • Scene context is king: The most significant insight is that a "one-size-fits-all" update strategy for visual trackers is insufficient. Adapting the update mechanism based on the specific challenges of the current scene (e.g., occlusion vs. deformation) dramatically improves robustness.
  • Balancing speed and accuracy: The paper demonstrates that it's possible to achieve accurate scale estimation and adaptive updating without crippling the tracker's speed, a common trade-off in many tracking methods.
  • Feature fusion for scene classification: Combining similarity, holistic response, and block response provides a rich set of cues to accurately classify the target's current scene, enabling the adaptive updating mechanism.

Key Results

The AKCF algorithm shows significant improvements over the KCF tracker and other state-of-the-art methods on the CVPR2013 benchmark. Specifically, it achieved improvements of:

  • 33.3% in scale variation scenes
  • 15% in partial or long-time large-area occlusion
  • 6% in deformation
  • 21.9% in fast motion
  • 19.8% in out-of-view scenes

Qualitative results show the AKCF tracker maintaining accurate bounding boxes and scales in challenging sequences where other algorithms drift or fail. The average tracking speed is 26.17 fps, which is competitive, especially considering its scale estimation and adaptive updating capabilities.

Limitations and Future Work

  • Simultaneous deformation and occlusion: The authors assume that deformation and occlusion do not occur simultaneously during tracking, as this scenario is considered too difficult with their current approach. This is a clear area for future work.
  • Parameter tuning: The algorithm involves several thresholds for scene classification and adaptive updating. While the paper discusses threshold selection and provides experimental validation, these parameters might still require careful tuning for optimal performance in diverse scenarios.
  • Computational cost of block response: While the overall scale estimation is low-complexity, the block response calculation for scene classification might add some computational overhead, though it's managed by limiting the number of blocks.

The paper concludes by highlighting the effectiveness of their scene-aware adaptive updating mechanism and low-complexity scale estimation, suggesting that this approach can significantly enhance visual tracking performance in challenging real-world conditions.

Summary updated

Abstract

In recent years, visual object tracking has been widely used in military guidance, human-computer interaction, road traffic, scene monitoring and many other fields. The tracking algorithms based on correlation filters have shown good performance in terms of accuracy and tracking speed. However, their performance is not satisfactory in scenes with scale variation, deformation, and occlusion. In this paper, we propose a scene-aware adaptive updating mechanism for visual tracking via a kernel correlation filter (KCF). First, a low complexity scale estimation method is presented, in which the corresponding weight in five scales is employed to determine the final target scale. Then, the adaptive updating mechanism is presented based on the scene-classification. We classify the video scenes as four categories by video content analysis. According to the target scene, we exploit the adaptive updating mechanism to update the kernel correlation filter to improve the robustness of the tracker, especially in scenes with scale variation, deformation, and occlusion. We evaluate our tracker on the CVPR2013 benchmark. The experimental results obtained with the proposed algorithm are improved by 33.3%, 15%, 6%, 21.9% and 19.8% compared to those of the KCF tracker on the scene with scale variation, partial or long-time large-area occlusion, deformation, fast motion and out-of-view.
BibTeX
@article{li2017scene,
  author = {Fan Li and Sirou Zhang and Xiaoya Qiao},
  title = {{Scene-Aware Adaptive Updating for Visual Tracking via Correlation Filters}},
  journal = {Sensors},
  year = {2017},
  doi = {10.3390/s17112626},
}
Figure