Real-Time Tracking by Double Templates Matching Based on Timed Motion History Image with HSV Feature

Zhiyong Li, Pengfei Li, Xiaoping Yu, Mervat HashemThe Scientific World JOURNAL 2014 · DOI

TL;DR

This paper introduces a novel real-time object tracking method that uses a combination of a timed motion history image (tMHI) and HSV color histograms to create robust appearance models. By employing a double template matching strategy (online and offline templates), the method effectively handles challenges like scale variation, pose changes, illumination shifts, and occlusions, outperforming existing trackers in various scenarios.

What Problem Does This Paper Address?

Tracking moving objects in real-world scenarios is a notoriously difficult problem due to several factors: changes in object scale, variations in pose (how the object is oriented), fluctuating illumination conditions, and temporary or complete occlusions. Traditional tracking algorithms often struggle to adapt to these dynamic changes, leading to tracking drift or complete loss of the target. The core challenge lies in creating an appearance model for the target that is both descriptive enough to distinguish it from the background and robust enough to withstand these environmental and object-based variations, all while maintaining real-time performance.

Methods

The proposed method, termed tMHI-HSV-based DTM tracking, combines several key techniques:

  • Timed Motion History Image (tMHI): This is an extension of the traditional Motion History Image (MHI). While MHIs capture motion information, tMHIs add a temporal decay mechanism, giving more weight to recent motion. This helps in segmenting moving objects and creating motion silhouettes.
  • HSV Color Histogram Feature: To overcome the limitations of relying solely on motion silhouettes, the authors incorporate HSV color histograms. HSV (Hue, Saturation, Value) is chosen over RGB because it better represents human perception of color and is more robust to illumination changes. The HSV color histogram of candidate object patches is used as a feature descriptor.
  • tMHI-HSV Representation: The tMHI is used to segment motion regions, and then the HSV color histogram of these regions is calculated. This combined feature is used to represent candidate object patches and templates.
  • Double Template Matching (DTM): This is a crucial innovation. Instead of relying on a single template, the method uses two:
  • Online Template: This template is updated in real-time with the current best estimate of the target's appearance. This allows for quick adaptation to minor changes.
  • Offline Template: This template is more stable and is updated less frequently. It serves as a fallback to prevent "overfitting" of the online template to noisy or rapidly changing environments. The target is considered correctly tracked if it matches well with both templates.
  • Template Updating: The online template is updated with the best candidate patch in each frame. The offline template is updated only when its appearance significantly differs from the online template, controlled by a threshold. This strategy balances adaptability with stability.

Datasets

The paper evaluates its method on several video sequences:

  • PETS2000 sequences: Specifically, "blue car," "white car," and "girl pose." These are used to test scale variation and general tracking performance.
  • Intelligent room sequence: Used to evaluate performance under pose change and scale variation.
  • Girl occlusion sequence: Specifically designed to test robustness against heavy and complete occlusions.
  • Car sequence: Used to assess performance under significant illumination changes.

The paper does not introduce new datasets but uses existing ones for comparative evaluation.

Key Insights and Ideas

  • HSV is better than RGB for illumination robustness: The choice of HSV color space for feature representation is a key insight, as it inherently handles illumination variations better than RGB, which is crucial for real-world tracking.
  • Motion + Color = Robustness: Combining motion information (tMHI) with color features (HSV histogram) provides a more comprehensive and discriminative representation of the target than either feature alone.
  • The power of two templates: The double template matching strategy (online and offline) is a significant contribution. It elegantly addresses the trade-off between rapid adaptation (online template) and stability against noise and rapid environmental changes (offline template), preventing drift and tracking loss.
  • tMHI's temporal aspect is key: The timed aspect of tMHI, giving more weight to recent motion, helps in segmenting moving objects more effectively and creating a more relevant motion history.

Key Results

  • Quantitative:
    • The proposed method achieves the highest success rates (SR) across various challenging sequences, often significantly outperforming OBT (Online Boosting Tracker) and CT (Compressive Tracking) methods. For example, in the "Blue car" sequence, the SR is 98% compared to 76% for OBT and 80% for CT.
  • Center Location Error (CLE) is consistently lower for the proposed method, indicating more precise tracking.
  • The tracker achieves real-time performance, with frame rates reaching up to 62 FPS for smaller video resolutions (320x240).
  • Qualitative:
    • The method demonstrates strong performance in handling scale variation, pose changes, and illumination changes, as visually confirmed in the provided figures.
  • It shows superior robustness to occlusion, successfully tracking targets even when they are heavily or completely obscured, where other methods fail.

Limitations and Future Work

  • Limitations:
    • The paper acknowledges that the current method is designed for fixed camera scenarios.
  • While HSV is robust to illumination, extreme and rapid changes might still pose challenges.
  • Future Work:
    • Extend the method to work with moving cameras.
  • Explore the integration of other features beyond HSV color histograms to further enhance tracking capabilities.

Summary updated

Abstract

It is a challenge to represent the target appearance model for moving object tracking under complex environment. This study presents a novel method with appearance model described by double templates based on timed motion history image with HSV color histogram feature (tMHI-HSV). The main components include offline template and online template initialization, tMHI-HSV-based candidate patches feature histograms calculation, double templates matching (DTM) for object location, and templates updating. Firstly, we initialize the target object region and calculate its HSV color histogram feature as offline template and online template. Secondly, the tMHI-HSV is used to segment the motion region and calculate these candidate object patches' color histograms to represent their appearance models. Finally, we utilize the DTM method to trace the target and update the offline template and online template real-timely. The experimental results show that the proposed method can efficiently handle the scale variation and pose change of the rigid and nonrigid objects, even in illumination change and occlusion visual environment.
BibTeX
@article{li2014real,
  author = {Zhiyong Li and Pengfei Li and Xiaoping Yu and Mervat Hashem},
  title = {{Real-Time Tracking by Double Templates Matching Based on Timed Motion History Image with HSV Feature}},
  journal = {The Scientific World JOURNAL},
  year = {2014},
  doi = {10.1155/2014/793769},
}
Figure