Instance significance guided multiple instance boosting for robust visual tracking

Jinwu Liu, Yao Lu, Tianfei Zhou

Abstract

Multiple Instance Learning (MIL) recently provides an appealing way to alleviate the drifting problem in visual tracking. Following the tracking-by-detection framework, an online MILBoost approach is developed that sequentially chooses weak classifiers by maximizing the bag likelihood. In this paper, we extend this idea towards incorporating the instance significance estimation into the online MILBoost framework. First, instead of treating all instances equally, with each instance we associate a significance-coefficient that represents its contribution to the bag likelihood. The coefficients are estimated by a Bayesian formula that jointly considers the predictions from multiple randomized MILBoost classifiers. Next, we incorporate the estimates within a new boosting procedure for more effectively selection of weak classifiers. Experiments with challenging public datasets show that the proposed method outperforms both existing MIL based and boosting based trackers.

Introduction

Most modern visual trackers operate within a "tracking-by-detection" framework. In this paradigm, the target is treated as a positive example, while the surrounding background provides negative examples used to train a classifier on the fly. However, this approach is susceptible to model drift. If the tracker's bounding box is even slightly misaligned, it may begin training on "dirty" data—samples containing both the target and the background. Over time, these small errors accumulate, causing the tracker to eventually lose the target entirely.

Multiple Instance Learning (MIL) was introduced to mitigate this by using "bags" of samples rather than single labeled instances. A positive bag contains several image patches near the target, assuming that at least one is a high-quality match. However, standard MIL approaches, such as the popular MILBoost, typically treat every patch within a bag as equally important. In dynamic environments involving fast motion, occlusions, or pose changes, many patches in a positive bag are actually noisy. Treating them with equal weight can degrade the classifier's performance.

Proposed Methodology: Instance Significance Guided MILBoost

To address the limitations of standard MIL, this paper introduces Instance Significance Guided MILBoost. This method moves beyond the "all samples are equal" assumption by intelligently weighting individual image patches based on their relevance to the target.

1. Significance-Coefficients ($r_{ij}$)

Instead of uniform weighting, each instance (patch) in a bag is assigned a significance-coefficient. This coefficient represents the instance's specific contribution to the overall bag likelihood, allowing the model to prioritize the most representative samples.

2. Ensemble Estimation and Bayesian Integration

To calculate these coefficients robustly, the system employs the "wisdom of the randomized crowd." It trains an ensemble of $N$ randomized MILBoost learners, each exposed to a slightly different subset of negative data. The significance of a patch is then determined using a Bayesian formula that aggregates predictions from the entire ensemble. If the majority of randomized learners agree that a patch represents the target, its significance score increases.

3. Refined Noisy-OR Model

The authors modify the standard "Noisy-OR" probability model—the mathematical foundation for determining bag positivity—by incorporating the significance coefficients into the exponent. This ensures that highly significant patches exert a much stronger influence on the final classifier's training than noisy or ambiguous ones.

4. Boosting Selection

These significance-weighted samples are used within a boosting procedure to select the most discriminative "weak classifiers" (utilizing Haar-like features). These are then combined to form a robust, strong final detector capable of handling complex visual changes.

Experimental Evaluation

The proposed tracker was evaluated against six state-of-the-art algorithms, including ASLA, PCOM, and the original MILBoost, across nine challenging public image sequences totaling over 9,000 frames. These sequences (e.g., Boy, Dog, Doll, Panda, Girl) were selected for their specific challenges, such as high motion blur, low resolution, and significant pose or scale changes.

Qualitative comparison of the proposed tracker against state-of-the-art methods across several challenging video sequences.

Key Results

The experimental results demonstrate that the proposed method significantly outperforms existing boosting-based and MIL-based trackers in both accuracy and stability.

Quantitative Performance

As shown in the performance table below, the proposed method achieved superior results across nearly all test cases:

  • Center Location Error (CLE): The proposed method achieved an average error of 7.8 pixels, outperforming the next best competitor, ASLA (8.2 pixels), and vastly improving upon the original MIL (22.5 pixels).
  • VOC Overlap Rate (VOR): The method achieved a 0.75 overlap score, compared to 0.71 for ASLA and 0.53 for the original MIL.
Quantitative comparison using Center Location Error (CLE) and VOC Overlap Rate (VOR) across nine public datasets.

Qualitative and Operational Insights

Qualitatively, the tracker showed remarkable stability in sequences like "Boy" and "Doll," which involve heavy motion blur that typically causes other trackers to drift. Operationally, despite the ensemble-based approach, the MATLAB implementation maintains a speed of 15 frames per second on a standard i7 CPU, making it suitable for near real-time applications.

Conclusion and Future Work

By incorporating instance significance into the MIL framework, this research provides a robust solution to the drifting problem in visual tracking. The use of randomized ensembles to estimate significance proves more effective than simple geometric weighting.

Future work may address the current limitation of a fixed bounding box scale, as the current model does not explicitly resize for objects moving toward or away from the camera. Additionally, integrating modern deep-learning-based features could further enhance the discriminative power of the significance-guided framework.

Summary updated

BibTeX
@inproceedings{liu2016instance,
  author = {Jinwu Liu and Yao Lu and Tianfei Zhou},
  title = {{Instance significance guided multiple instance boosting for robust visual tracking}},
  booktitle = {ICIP},
  year = {2016},
  doi = {10.1109/icip.2016.7532647},
}
Figure