Enhancement of ELDA Tracker Based on CNN Features and Adaptive Model Update

Changxin Gao, Huizhang Shi, Jin-Gang Yu, Nong Sang

TL;DR

This paper enhances the ELDA visual tracking algorithm by integrating deep Convolutional Neural Network (CNN) features for improved appearance representation and introducing adaptive model updates for better robustness. The proposed ECT tracker achieves state-of-the-art performance on a challenging benchmark dataset, demonstrating its effectiveness in complex tracking scenarios.

What Problem Does This Paper Address?

Visual tracking, the process of following an object across video frames, is crucial for applications like surveillance and robotics. A common and effective approach is "tracking-by-detection," which frames tracking as a detection problem. However, these methods rely heavily on accurate appearance representation and robust observation models. Existing methods, like the Exemplar-based Linear Discriminant Analysis (ELDA) tracker, face challenges in balancing discriminative power (distinguishing the object from background) and adaptivity (handling changes in object appearance and environment). Specifically, traditional features like HOG can be less discriminative than modern deep learning features, and existing model update strategies can be inefficient or insufficient for complex scenes. This paper aims to address these limitations by leveraging the power of CNNs and developing more adaptive model update mechanisms.

Methods

The core of the proposed enhancement lies in several key modifications to the ELDA framework:

  • CNN Features for Appearance Representation: Instead of using Histogram of Oriented Gradients (HOG) features, the paper integrates deep CNN features. To overcome the computational cost of extracting CNN features from every candidate window, a novel "two-step" approach is proposed. This involves pre-computing convolutional layer features (conv5) across the entire image region and then computing fully-connected layer features (fc7) only on specific patches within these feature maps. This significantly speeds up feature extraction.
  • Adaptive Object Model Update: The ELDA tracker traditionally uses a fixed-size window of past object models for its short-term model. The proposed method improves this by employing a greedy strategy to select "good" object models. These models are chosen based on their discriminative ability (high confidence score on the current frame) and their low correlation with already selected models, leading to a more compact and effective object model.
  • Gaussian Mixture Model (GMM) for Background: ELDA uses a single Gaussian model for the background, which is insufficient for complex scenes. The paper replaces this with a Gaussian Mixture Model (GMM), which can better represent complex background distributions and is updated online to adapt to changing environments.

Datasets

The primary dataset used for evaluation is the CVPR2013 benchmark dataset. This dataset comprises 50 video sequences covering 11 challenging tracking scenarios, including illumination variation, scale variation, occlusion, deformation, motion blur, fast motion, in-plane rotation, out-of-plane rotation, out-of-view, background clutter, and low resolution. The paper does not introduce a new benchmark dataset.

Key Insights and Ideas

  • CNNs are a Game Changer for Tracking: The paper strongly demonstrates that deep CNN features, even when pre-trained on general datasets, offer significantly better discriminative power for visual tracking compared to traditional features like HOG. This highlights the potential of leveraging large-scale pre-trained deep models for various vision tasks.
  • Efficiency is Key for Deep Features in Tracking: The "two-step" CNN feature extraction method is crucial. Without it, the computational cost of using deep features would likely make real-time tracking infeasible. This approach cleverly balances feature richness with computational efficiency.
  • Adaptive Background Modeling Matters: The shift from a single Gaussian to a Gaussian Mixture Model for background representation is shown to be critical for handling complex and dynamic scenes, improving the tracker's robustness.
  • Selective Model Updates are More Effective: The greedy approach to selecting object models based on discriminative ability and low correlation leads to a more compact and robust short-term object model, outperforming simpler window-based selection.

Key Results

The proposed ECT tracker achieved state-of-the-art performance on the CVPR2013 benchmark dataset.

  • Overall Performance: The paper reports that ECT achieved the best overall performance across both precision and success rate metrics on the benchmark. Specifically, it achieved a precision of 0.847 at a threshold of 20 pixels and a success rate of 0.605 at an overlap threshold of 0.2.
  • Attribute-Specific Performance: ECT also demonstrated superior performance across most of the 11 challenging attributes, particularly excelling in scenarios with deformations, fast motions, background clutters, and low resolution.
  • Speed: The tracker runs at approximately 2 frames per second (fps) on a workstation with a GPU, and about 0.6 fps on CPU only. This is considered efficient given the use of deep CNN features.
  • Component Analysis: Ablation studies showed that using CNN features (ELDA_CNN) provided the most significant performance boost over the original ELDA. The adaptive background model (ECT-BM) also contributed substantially, while the compact object model (ECT-OM) offered a moderate improvement.

Limitations and Future Work

  • Scale Variation: While ECT performs very well, it did not rank first in the success plot for scale variation. The authors attribute this to the detectors being designed for a fixed scale.
  • Pre-trained Networks: The current CNN architecture is pre-trained and not specifically fine-tuned for tracking individual object instances. The authors suggest that refining the network during tracking, with low time cost, could further improve performance. This is identified as a key direction for future work.

Summary updated

BibTeX
@inproceedings{gao2016enhancement,
  author = {Changxin Gao and Huizhang Shi and Jin-Gang Yu and Nong Sang},
  title = {{Enhancement of ELDA Tracker Based on CNN Features and Adaptive Model Update}},
  booktitle = {Sensors},
  year = {2016},
  url = {https://openreview.net/forum?id=i4L5AmEYkmSy},
}
Figure