Progressive Multi-Stage Learning for Discriminative Tracking

Weichao Li, Xi Li, Omar El Farouk Bourahla, Fuxian Huang, Fei Wu, Wei Liu, Zhiheng Wang, Hongmin Liu

TL;DR

This paper introduces a novel "progressive multi-stage learning" approach for visual tracking that tackles the challenge of adapting to changing object appearances by intelligently selecting training samples. By progressively learning from easy to hard samples, guided by both tracking loss and detection confidence, their method achieves more robust and accurate tracking, especially in challenging scenarios.

What Problem Does This Paper Address?

Visual tracking aims to follow a specific object across a sequence of images. A key challenge is adapting the tracker's internal model to the object's changing appearance due to factors like illumination changes, occlusions, and shape deformations. Online learning is crucial, but it relies on training samples derived from the tracker's own predictions, which can be noisy or incorrect. This can lead to "error accumulation" and "model drift," where the tracker loses the target. Existing methods often struggle to reliably select good training samples, especially when dealing with significant intra-class variations (the object looking very different) while maintaining inter-class separability (distinguishing the object from the background).

Methods

The core innovation is a "progressive multi-stage learning" strategy. Instead of a single-stage update, the learning process is divided into multiple stages, each with an increasing "learning pace" (λ). This allows the tracker to first learn from the most reliable, "easy" samples and then gradually incorporate more difficult ones.

Key components of their method include:

  • Joint Discriminative Learning and Sample Selection: The appearance model and sample weights are optimized simultaneously.
  • Self-Paced Learning: Inspired by human learning, samples are introduced incrementally based on their difficulty (measured by loss).
  • Multi-Stage Optimization: The learning process is broken down into N stages, with the learning pace λ increasing at each stage. This allows for an "easy-to-hard" sample selection policy.
  • Time-Weighted Sample Importance (p_k): Recent samples are given more weight, reflecting the temporal nature of tracking.
  • Detection Confidence Integration (c_k): Beyond just tracking loss, the quality of the object detection response map is used to assess sample reliability. A sharp, single peak in the response map indicates higher confidence.
  • Adaptive Convex Search (ACS): Used to efficiently solve the biconvex optimization problem arising from the joint learning.

Datasets

The paper evaluates their approach on several standard visual tracking benchmarks:

  • OTB-2015: A widely used dataset for evaluating tracking algorithms.
  • Temple Color Dataset: Another benchmark dataset.
  • VOT-2015, VOT-2016, VOT-2017: Datasets from the Visual Object Tracking challenge, known for their challenging sequences.

Key Insights and Ideas

  • Progressive Learning is Key: Breaking down sample selection into stages, from easy to hard, is crucial for handling complex appearance variations and preventing early corruption of the model by noisy samples.
  • Beyond Just Loss: Relying solely on tracking loss for sample selection is insufficient. Incorporating detection confidence provides a more robust measure of sample quality.
  • Temporal Dynamics Matter: Giving more weight to recent samples is essential for adapting to the dynamic nature of object appearance in video sequences.
  • Multi-Stage vs. One-Stage: The multi-stage approach demonstrably outperforms a single-stage self-paced learning strategy, especially in challenging scenarios.

Key Results

  • Improved Accuracy: The proposed method, particularly the version incorporating both temporal importance and detection confidence (SPL-DCI), achieves significant improvements in Precision (PS) and AUC scores on the OTB-2015 dataset compared to baseline trackers and even its own simpler variants. For instance, it achieved a final relative gain of 5.8% in PS and 6.4% in AUC over the baseline.
  • Competitive Performance: The tracker demonstrates competitive performance against state-of-the-art methods, including those using hand-crafted features and deep features, across various challenging attributes like occlusion, scale variation, and illumination changes.
  • Robustness: The method shows strong performance on the VOT datasets, outperforming baselines in accuracy, robustness, and expected average overlap (EAO).
  • Efficiency: While not the primary focus, the method is integrated with efficient trackers like SRDCF, and the optimization process is designed to be computationally feasible.

Limitations and Future Work

  • Parameter Sensitivity: The paper notes that the initial learning pace (λ) and scale ratio (μ) are important and ideally should be adapted to each video sequence, suggesting potential for further research in automatic parameter tuning.
  • Computational Cost: While efficient, the multi-stage optimization inherently adds some computational overhead compared to simpler single-stage methods.
  • Future Work: The authors suggest exploring more sophisticated self-paced controllers that can incorporate additional prior information for even better sample selection and tracking performance. They also highlight the potential for further improvements by integrating more advanced deep features.

Summary updated

BibTeX
@article{li2022progressive,
  author = {Weichao Li and Xi Li and Omar El Farouk Bourahla and Fuxian Huang and Fei Wu and Wei Liu and Zhiheng Wang and Hongmin Liu},
  title = {{Progressive Multi-Stage Learning for Discriminative Tracking}},
  journal = {IEEE Trans. Cybernetics},
  year = {2022},
  url = {https://openreview.net/forum?id=n6UR9WEIVTl},
}
Figure