State-Aware Anti-Drift Object Tracking

Yuqi Han, Chenwei Deng, Baojun Zhao, Dacheng TaoIEEE TIP 2019 · DOI

TL;DR

This paper introduces a novel "State-aware Anti-drift" (SAT) correlation filter-based tracker that jointly models target discrimination and reliability to combat model drift. By incorporating contextual information and a learned reliability mask, SAT significantly improves tracking robustness against external distractions and internal appearance variations.

What Problem Does This Paper Address?

Visual tracking, the task of estimating an object's position and size in a video sequence, is crucial for many applications. While correlation filter (CF) based trackers offer a good balance of accuracy and speed, they often suffer from "model drift." This occurs when the tracker's learned model deviates from the true target due to factors like:

  • External Distractions: Similar-looking objects or background clutter that confuse the tracker.
  • Internal Appearance Variations: Changes in the target's shape, pose, or illumination that make the learned model less representative.

Existing CF trackers often struggle to handle these issues simultaneously, leading to tracking failures. This paper aims to address this by developing a tracker that is "state-aware," meaning it can adapt its tracking strategy based on the current tracking conditions, specifically by considering both the target's distinctiveness from its surroundings and the reliability of the current tracking estimate.

Methods

The core of the proposed SAT tracker lies in a novel optimization problem for learning the correlation filter. It extends traditional CF trackers by:

  • Discrimination Modeling: Incorporating surrounding contextual patches into the filter training. This helps the tracker learn to distinguish the target from potential distractors by explicitly training on both the target and its immediate background. The regression labels for these context patches are set to zero, effectively treating them as negative samples.
  • Reliability Modeling: Learning a color-based reliable mask for each frame. This mask assigns a probability of belonging to the foreground (target) or background for each pixel within the bounding box. This is achieved by modeling color histograms of foreground and background regions and using a color informativeness test to ensure the mask is reliable. The final filter is then an element-wise product of the discriminative filter and this reliability mask.
  • High-Confidence Filter Updating: Introducing a "state-aware" updating strategy. Instead of updating the filter every frame, SAT monitors the tracking condition using two metrics: the maximum response score ($S_{max}$) and the kurtosis ($BK$) of the response map. Kurtosis measures the "peakedness" and tail weight of the response distribution, indicating how well-defined the target's location is. The filter is only updated if both metrics exceed learned thresholds, preventing updates based on noisy or ambiguous responses and thus avoiding contamination of the learned filter.
  • Optimization: The proposed optimization problem is solved efficiently using the Alternating Direction Method of Multipliers (ADMM) in the frequency domain, which avoids computationally expensive matrix inversions.

The paper also proposes a "DeepSAT" variant that leverages pre-trained CNN features (from VGG-Net) for enhanced feature representation, further improving performance.

Datasets

The proposed tracker is evaluated on the OTB-100 dataset, which is a standard benchmark for visual tracking. The paper does not introduce a new dataset.

Key Insights and Ideas

  • Joint Discrimination and Reliability: The most significant insight is that robust tracking requires simultaneously considering how well the target is distinguished from its background and how reliable the current tracking estimate is.
  • Context is Key for Distractors: Explicitly incorporating surrounding context during filter training is crucial for distinguishing the target from similar distractors.
  • Color Reliability for Appearance Changes: A learned color-based mask effectively handles internal appearance variations (like rotation and deformation) by focusing the filter on more stable color cues.
  • Kurtosis as a Tracking Monitor: Using kurtosis of the response map is an effective way to assess the quality and confidence of the tracking result, going beyond just the peak response. This allows for a more intelligent and robust filter updating strategy.
  • State-Awareness Prevents Drift: By making the filter update conditional on the tracking quality (using $S_{max}$ and $BK$), the tracker becomes "state-aware," significantly reducing model drift.

Key Results

  • Quantitative Performance: The SAT tracker achieves state-of-the-art performance on the OTB-100 benchmark, outperforming 13 other trackers. It achieves an AUC score of 0.607 and a precision rate of 0.837 (at 20 pixels), improving upon the baseline SAMF-CA by 3.3% and 4.6% respectively. The DeepSAT variant also shows competitive results against other CNN-based trackers.
  • Attribute-Specific Performance: SAT consistently ranks among the top trackers across various challenging attributes, including background clutter, illumination variation, occlusion, deformation, in-plane and out-of-plane rotation, and scale variation.
  • Qualitative Results: Visualizations demonstrate SAT's ability to maintain tracking through challenging scenarios like occlusion and abrupt motion, where other trackers fail. The paper provides visual examples in Figure 8.

Limitations and Future Work

  • Computational Cost of CNN Features: While DeepSAT leverages CNN features, the paper notes that they can be computationally expensive, especially for scale estimation. Future work could explore more efficient CNN architectures or feature extraction methods.
  • Scale Estimation: The paper mentions that pre-trained CNN features have limited effectiveness for scale estimation, and they adopt strategies from other works. Further research into more integrated and efficient scale estimation within the SAT framework could be beneficial.
  • Generalization of Color Model: The color-based reliability mask might be sensitive to extreme illumination changes or targets with colors very similar to the background. While a color informativeness test is included, further exploration of more robust color modeling or alternative reliability cues could be explored.

Summary updated

BibTeX
@article{han2019state,
  author = {Yuqi Han and Chenwei Deng and Baojun Zhao and Dacheng Tao},
  title = {{State-Aware Anti-Drift Object Tracking}},
  journal = {IEEE Trans. Image Process.},
  year = {2019},
  doi = {10.1109/tip.2019.2905984},
}
Figure