IGSSTRCF: Importance Guided Sparse Spatio-Temporal Regularized Correlation Filters For Tracking

Monika Jain, A. Venkata Subramanyam, Simon Denman, Sridha Sridharan, Clinton FookesWACV 2021 · OpenReview

TL;DR

This paper introduces IGSSTRCF, a novel visual object tracking algorithm that explicitly models and regularizes changes in correlation filters and their spatial weights across frames. By incorporating sparsity penalties and adaptive channel importance, IGSSTRCF achieves more robust tracking and outperforms several state-of-the-art methods on benchmark datasets.

What Problem Does This Paper Address?

Visual object tracking is a critical task in computer vision, and Correlation Filter (CF) based trackers have become popular due to their efficiency. However, these trackers often struggle with issues like "tracker drift" caused by artificial negative samples and the inability to adapt to rapid appearance changes, occlusions, or background clutter. Existing methods that attempt to address these issues, such as those incorporating spatial or temporal constraints, often fail to adequately model the dynamic variations between consecutive frames or the varying importance of different feature channels. This can lead to degraded performance when the target's appearance or the background context changes significantly.

Methods

The core innovation of IGSSTRCF lies in its formulation of a novel Importance Guided Sparse Spatio-Temporal Regularization (IGSSTRCF) framework. This framework explicitly models:

  • Sparse Spatial Regularization (SSR): This component learns spatial weights that adapt to changes between consecutive frames. By penalizing sparse differences, it ensures that only relevant spatial variations are incorporated, suppressing the influence of background clutter and boundary effects.
  • Sparse Temporal Regularization (STR): This component models the sparse difference between the current and previous correlation filters. This allows the filter to adapt to appearance changes while preventing drift by only incorporating necessary updates.
  • Channel Importance (CI): An adaptive mechanism is introduced to assign importance weights to different feature channels. This is crucial because the informativeness of a channel can change over time. By prioritizing more informative channels and suppressing less useful ones, the tracker avoids degradation caused by irrelevant features.

The entire formulation is efficiently solved using the Alternating Direction Method of Multipliers (ADMM), a common optimization technique for such problems. The paper details the specific ADMM sub-problems for solving for the filter (H), feature channel weights (q), spatial weights (w), and sparse difference vectors (Bw, Bh).

Datasets

The proposed IGSSTRCF tracker was evaluated on the following benchmark datasets:

  • TC128 [36]
  • UAV123 [40]
  • VOT-2017 [26]
  • VOT-2019 [27]

Key Insights and Ideas

  • Explicitly modeling variations is key: The paper argues that simply enforcing similarity to previous states (filters or weights) isn't enough. Explicitly modeling the sparse difference between current and previous states allows for more nuanced adaptation to dynamic changes.
  • Sparsity as a regularization tool: Imposing sparsity on the differences in spatial weights and temporal filters is shown to be effective in filtering out noise and irrelevant information, leading to more robust learning.
  • Channel importance is dynamic: The insight that different feature channels contribute differently to tracking at different times is crucial. The adaptive channel importance mechanism directly addresses this, preventing less informative channels from negatively impacting performance.
  • ADMM is a powerful tool for complex formulations: The successful application of ADMM to solve the multi-component optimization problem highlights its utility in tackling complex tracking frameworks.

Key Results

The IGSSTRCF tracker demonstrated strong performance across multiple datasets:

  • Outperforms state-of-the-art: The paper reports that IGSSTRCF outperforms several state-of-the-art trackers on the TC128 and UAV123 datasets in terms of precision and success plots.
  • Competitive on VOT datasets: On the VOT-2019 and VOT-2017 datasets, IGSSTRCF achieved competitive results, particularly excelling in robustness and EAO (Expected Average Overlap) on VOT-2019, and being third best in robustness and EAO on VOT-2017.
  • Ablation studies confirm component contributions: An ablation study showed that each regularization component (SSR, STR, CI) contributes positively to the overall tracker performance, with the full IGSSTRCF formulation achieving the best results.
  • Qualitative success: Qualitative results showed IGSSTRCF successfully tracking through challenges like background clutter, similar object presence, and multi-object presence, though it, like other trackers, struggled with significant scale changes.

Limitations and Future Work

  • Limitations: While the paper demonstrates strong performance, it acknowledges that IGSSTRCF, like most trackers, can fail in extreme scenarios such as rapid scale changes or severe in-plane rotations (as seen in the qualitative evaluation on the "Charger_ce, skating_ce2, and MotorRolling" sequences). The paper also mentions that the performance on TC128 and UAV123 was slightly surpassed by GFSDCF and ECO in some metrics, suggesting room for further improvement.
  • Future Work: The paper doesn't explicitly detail future work directions, but the success of the proposed sparse spatio-temporal regularization and adaptive channel importance mechanisms suggests that exploring more sophisticated regularization techniques or integrating these ideas into other tracking paradigms could be fruitful avenues. Further investigation into handling extreme scale variations and rotations could also be a focus.

Summary updated

Abstract

This paper proposes a novel Importance Guided Sparse Spatio-Temporal Regularization based Correlation Filter (IGSSTRCF) tracker. Our formulation explicitly models the variations in the correlation filters and associated spatial weights in successive frames. By imposing a sparsity penalty on these variations, the formulation ensures that only relevant changes are incorporated during updates. This results in more robust filter coefficients that minimize the tracking drift. The IGSSTRCF also includes an adaptive channel importance estimation strategy that assigns an importance weight to each feature channel during training. The proposed formulation is efficiently solved via the alternating direction method of multipliers. A comparative analysis is shown on TC128, UAV123, VOT-2017, and VOT-2019 datasets; and we present an ablation study to demonstrate the contribution of each component of the IGSSTRCF. It is observed that we outperform several state-of-the-art trackers and each component of the proposed IGSSTRCF contributes positively towards tracker performance.
BibTeX
@inproceedings{jain2021igsstrcf,
  author = {Monika Jain and A. Venkata Subramanyam and Simon Denman and Sridha Sridharan and Clinton Fookes},
  title = {{IGSSTRCF: Importance Guided Sparse Spatio-Temporal Regularized Correlation Filters For Tracking}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2021},
  url = {https://openreview.net/forum?id=TgTBgBFw-JA},
}
Figure