A motion observable representation using color correlogram and its applications to tracking

Qi Zhao, Hai TaoCVIU 2009 · DOI

TL;DR

This paper introduces a novel object representation called the Simplified Color Correlogram (SCC) that captures both spatial and color information, enabling robust tracking of object position and orientation. By extending the mean shift algorithm to a joint translation-rotation space, the SCC-based tracker achieves real-time performance and outperforms traditional methods in challenging scenarios with object rotation and partial occlusions.

What Problem Does This Paper Address?

Accurate and robust object tracking in real-time is a fundamental challenge in computer vision. Existing methods often struggle with objects that rotate, change orientation, or experience partial occlusions. While color histograms are simple and efficient, they discard spatial information, making orientation estimation impossible. More complex representations that capture spatial details can be computationally expensive or sensitive to illumination changes. This paper addresses the need for an object representation that is both computationally efficient and capable of capturing orientation variations, thereby improving tracking robustness in dynamic environments.

Methods

The core contribution is the Simplified Color Correlogram (SCC). Unlike traditional color correlograms used in image retrieval, the SCC is tailored for tracking by fixing the distance and selecting specific directions (axes) for pixel pairs. This simplification makes it computationally efficient while retaining sensitivity to orientation changes.

The authors extend the mean shift algorithm to a joint translation-rotation domain. Instead of just estimating the object's position, the algorithm now simultaneously estimates its orientation. This is achieved by defining a similarity function based on the SCC that is sensitive to both spatial shifts and rotations. The optimization process involves searching in this combined space.

A key aspect is the motion observability analysis. The authors analyze the condition number of a matrix derived from the SCC to determine how well different SCC configurations (axis choices and pair distances) can observe translational and rotational motion. This analysis helps in selecting optimal SCC parameters for reliable tracking.

Datasets

The paper evaluates the proposed SCC-based tracker on several datasets:

  • Car-chasing sequence: A live video sequence used to demonstrate robustness to rotation, occlusion, and scale changes.
  • Person-cart sequence: Used to show robustness to non-rigid objects, orientation changes, and camera motion.
  • PETS 2001 dataset: A standard benchmark for evaluating tracking algorithms, particularly useful for testing orientation estimation capabilities.
  • PETS 2004 dataset: Used to evaluate tracking of multiple interacting objects with similar color distributions but different spatial arrangements.
  • Arm sequence: Used to demonstrate tracking of elongated objects.
  • Handset sequence: Used to show robustness to fast object rotation.
  • Multiple human parts sequences (Figs. 14 & 15): Demonstrates tracking of articulated objects.
  • Face sequence: Used to track human faces with head tilts.

Key Insights and Ideas

  • SCC as a bridge: The SCC effectively bridges the gap between simple color histograms and complex template-based methods, offering a good balance of spatial information, computational efficiency, and robustness.
  • Orientation is observable with SCC: By carefully selecting axes and pair distances, the SCC can be made sensitive to object orientation, a capability lacking in standard color histograms.
  • Motion observability is crucial: The paper highlights that not all SCC configurations are equally good at observing motion. Analyzing the condition number of the system matrix is a principled way to select optimal SCC parameters for stable tracking.
  • Joint translation-rotation tracking is feasible and efficient: Extending mean shift to the joint domain is shown to be effective and can achieve real-time performance, making it practical for many applications.
  • Two-axis SCC offers a good trade-off: While single-axis SCCs are faster, two orthogonal axes often provide a more stable and robust solution without a significant computational penalty.

Key Results

  • The SCC-based tracker successfully tracks objects with rotations, partial occlusions, and scale changes, outperforming standard mean shift (MS) trackers in challenging scenarios.
  • For the car-chasing sequence, the SCC tracker maintained focus throughout 2250 frames, while MS trackers lost track during turns or cluttered backgrounds.
  • Quantitative results on PETS 2001 data show significantly lower position and size errors for the SCC-based tracker compared to the MS tracker, especially for sequences involving object rotation.
  • The algorithm achieves real-time performance, running at approximately 30 frames per second on a 3.2 GHz PC with a C++ implementation, which is comparable to standard MS trackers.
  • The analysis of condition numbers provides a method to select optimal SCC parameters, leading to more stable tracking.

Limitations and Future Work

  • Degenerate patterns: The paper identifies certain image patterns (e.g., concentric circles, parallel stripes) that can lead to unobservable motion in specific SCC configurations. While multi-axis SCCs mitigate this, it remains a theoretical limitation.
  • Computational cost: While real-time, the SCC-based tracker is computationally more expensive than standard MS due to the additional orientation estimation. The authors suggest minimizing the number of axes and directions considered.
  • Parameter tuning: The selection of optimal axes and pair distances, while analyzed, might still require some tuning for specific applications.
  • Future work: The authors suggest exploring more complex SCC configurations and integrating the SCC representation into other tracking frameworks beyond mean shift. The potential of SCC for other computer vision tasks like human behavior analysis is also hinted at.

Summary updated

BibTeX
@article{zhao2009motion,
  author = {Qi Zhao and Hai Tao},
  title = {{A motion observable representation using color correlogram and its applications to tracking}},
  journal = {CVIU},
  year = {2009},
  doi = {10.1016/j.cviu.2008.10.007},
}
Figure