Robust Joint Discriminative Feature Learning for Visual Tracking

Xiangyuan Lan, Shengping Zhang, Pong C. YuenIJCAI 2016

Abstract: Visual tracking remains a significant challenge due to dynamic appearance changes, occlusions, and cluttered backgrounds. This paper introduces a unified framework for "Robust Joint Discriminative Feature Learning" that bridges generative and discriminative modeling. By decomposing raw features into low-rank (intrinsic) and sparse (noise) components while simultaneously training classifiers, the proposed method achieves superior robustness. Experimental results on 15 benchmark sequences demonstrate that the framework significantly outperforms state-of-the-art trackers in both accuracy and success rate, particularly in scenarios involving heavy occlusion and complex backgrounds.

Introduction

Visual tracking—the task of estimating the trajectory of a target object in a video sequence—is a cornerstone of computer vision. However, its practical application is often hindered by factors such as illumination variation, partial or full occlusion, and background clutter.

Historically, researchers have relied on two distinct paradigms:

  1. Generative Models: These focus on accurately representing the object's appearance. While effective at handling appearance changes, they are easily distracted by cluttered backgrounds.
  2. Discriminative Models: These treat tracking as a classification problem (Target vs. Background). They handle clutter well but often fail if the features they are trained on become corrupted by shadows or obstacles.

The Proposed Framework

The researchers propose a unified optimization objective that performs three tasks simultaneously to achieve the "best of both worlds":

  • Robust Feature Learning: Inspired by Robust Principal Component Analysis (RPCA), the model decomposes raw features into a low-rank component (the "clean" intrinsic appearance) and a sparse component (the "noise" or corruption, such as an occluding branch).
  • Joint Discriminative Learning: Unlike traditional methods that extract features and then train a classifier, this framework learns the classifiers during the feature learning process. This ensures the "clean" features are specifically optimized to distinguish the target from the background.
  • Consensus-based Fusion: Instead of simple concatenation, the model introduces a "consensus classification score." It encourages different features (e.g., color and texture) to agree on the target's identity, while allowing for dynamic weighting if one feature becomes unreliable in a specific frame.

Experimental Evaluation

The model was evaluated on 15 challenging video sequences, including DavidIndoor (illumination), Faceocc2 (occlusion), and Bolt (fast motion and clutter).

Quantitative comparison of center location error across four benchmark sequences: Trellis, Skating1, DavidOutdoor, and Bolt. The proposed method (red line) consistently maintains the lowest error.

As shown in the quantitative plots, the proposed method demonstrates exceptional stability. In the "Bolt" sequence, which features a runner in a crowded environment, the tracker maintained a success rate of 0.75, whereas competing methods like STRUCK and MIL saw performance drop to near zero (0.01–0.02).

Qualitative tracking results on various sequences involving occlusion, lighting changes, and background clutter. The red bounding box represents the proposed method.

Qualitative analysis confirms these findings. In sequences like "Faceocc2," where the target is frequently occluded, the joint learning approach prevents the "drifting" problem that typically causes trackers to lose the target.

Conclusion and Future Work

The Robust Joint Discriminative Feature Learning framework effectively addresses the limitations of traditional tracking paradigms by cleaning features before classification and fusing multiple cues through consensus. While the current implementation runs at approximately 3 FPS due to the iterative nature of the optimization, its robustness sets a new benchmark for challenging scenarios. Future research will focus on improving computational speed and integrating deep learning features to further enhance extraction quality.

Summary updated

BibTeX
@inproceedings{lan2016robust,
  author = {Xiangyuan Lan and Shengping Zhang and Pong C. Yuen},
  title = {{Robust Joint Discriminative Feature Learning for Visual Tracking}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2016},
}
Figure