Combining Deep Learning and Preference Learning for Object Tracking

Shu-Chao Pang, Juan José del Coz, Zhezhou Yu, Oscar Luaces, Jorge DíezICONIP 2016 · DOI

TL;DR

This paper introduces DPL, a novel object tracking method that synergistically combines deep learning for robust feature extraction with preference learning for effective object localization. By framing tracking as a ranking problem, DPL achieves competitive performance against state-of-the-art algorithms, offering a new perspective on tackling this challenging computer vision task.

What Problem Does This Paper Address?

Object tracking, the task of following a moving object through a video sequence, is a fundamental problem in computer vision with applications ranging from surveillance to augmented reality. Despite significant progress, robustly tracking objects remains challenging due to real-world complexities like illumination variations and occlusions. Existing trackers often rely on manually chosen features or complex update strategies. This paper aims to improve tracking robustness by leveraging the power of deep learning for feature representation and preference learning for more effective object detection and localization.

Methods

The proposed DPL (Deep & Preference Learning) algorithm integrates two key learning frameworks:

  • Deep Learning (SDAE): A Stacked Denoising Autoencoder (SDAE) is used to automatically learn rich, multi-dimensional feature representations from raw image data. This deep learning architecture, composed of an encoder and a decoder, learns to compress images into a lower-dimensional feature space and then reconstruct them, effectively capturing discriminative visual information.
  • Preference Learning (SVM): Object tracking is reframed as a ranking problem. Given a set of candidate bounding boxes in a frame, preference learning is used to rank them based on their likelihood of containing the target object. Specifically, the algorithm learns a ranking function using Support Vector Machines (SVMs) trained on preference judgments derived from pairs of positive (near the target) and negative (far from the target) examples. This approach aims to ensure that regions highly overlapping with the target are ranked higher than others.

The DPL algorithm operates in four main steps: selecting positive and negative examples, extracting deep features using the SDAE, training the preference learning model, and finally, using a particle filter combined with the learned ranking model to detect the object in subsequent frames. A model update mechanism is also incorporated to handle drifting situations and appearance changes.

Datasets

The paper does not explicitly mention specific datasets used for training the SDAE model. However, for evaluation, they used 14 videos selected from a benchmark dataset (likely the one referenced in [9]) covering a range of difficulties (hard, middle, easy). These videos were used to compare DPL against 14 other state-of-the-art trackers.

Key Insights and Ideas

  • Tracking as a Ranking Problem: Reframing object tracking as a ranking problem, where candidate regions are ranked by their overlap with the target, is a novel and effective approach. This perspective allows preference learning to directly address the core task of identifying the most likely object location.
  • Synergy of Deep Learning and Preference Learning: The combination of deep learning for powerful feature extraction and preference learning for robust ranking proves to be a potent strategy. Deep learning provides the rich representations, while preference learning leverages these representations to make informed decisions about object localization.
  • Robustness through Ranking: The preference learning approach, by focusing on relative rankings rather than absolute scores, can be more robust to variations and noise, as it only requires that correct regions are ranked higher than incorrect ones.

Key Results

The DPL algorithm demonstrated competitive performance against 14 state-of-the-art object tracking methods.

  • Success Rate: DPL achieved a high success rate, particularly at lower overlap thresholds, indicating its ability to accurately detect at least part of the object.
  • Precision: While other trackers showed slightly better precision at very small location error thresholds, DPL significantly outperformed them when the error threshold increased. This suggests DPL is a robust tracker that rarely loses track of the object.
  • Statistical Analysis: A Nemenyi test indicated that DPL, along with SCM, performed significantly better than some of the weaker trackers. Although many differences were not statistically significant, the overall results were described as "quite promising."

Limitations and Future Work

  • Limitations: The paper notes that while DPL performs well, there is still room for improvement in accuracy. Specifically, its performance decays at higher overlap thresholds, suggesting potential for improvement in capturing the entire object area more precisely.
  • Future Work: The authors suggest that the application of preference learning opens up new research avenues for object tracking. Further exploration of this paradigm could lead to even more advanced and robust tracking systems.

Summary updated

BibTeX
@inproceedings{pang2016combining,
  author = {Shu-Chao Pang and Juan José del Coz and Zhezhou Yu and Oscar Luaces and Jorge Díez},
  title = {{Combining Deep Learning and Preference Learning for Object Tracking}},
  booktitle = {ICONIP},
  year = {2016},
  doi = {10.1007/978-3-319-46675-0\_8},
}
Figure