Robust Object Tracking with a Hierarchical Ensemble Framework

Mengmeng Wang, Yong Liu

Abstract

Autonomous robots require accurate and robust object tracking for applications ranging from home security to navigation. This paper introduces a Hierarchical Ensemble Tracking (HET) framework that integrates individual pixel features, local patches, and holistic target models. By combining multiple ensemble models simultaneously, the framework achieves high robustness against appearance changes and occlusions. A bottom layer handles local patch matching, a middle layer utilizes holistic templates for generative searching, and a top layer employs an adaptive Kalman filter for temporal consistency. Experimental results on benchmark sequences demonstrate that the proposed tracker outperforms several state-of-the-art algorithms in challenging environments.

Introduction

Visual tracking is a cornerstone of robotics, yet it remains difficult in real-world "wild" environments. Robots frequently encounter four primary hurdles: occlusions, deformations (e.g., a walking human), fast motion, and background clutter. While many modern trackers rely on deep learning, these often require expensive GPU hardware and are too computationally heavy for the standard CPUs found on mobile robots. There is a critical need for tracking algorithms that balance high accuracy with real-time performance on constrained hardware.

Methodology: The Hierarchical Ensemble Framework

The proposed HET framework is organized into three distinct layers to provide multi-level robustness:

  1. Bottom Layer (Local Features): The tracker extracts multiple overlapping sub-patches from the target. It uses Compressive Sensing to project high-dimensional pixel features (such as Haar-like features) into a lower-dimensional space, ensuring computational efficiency. For each sub-patch, a pool of weak Naive Bayesian classifiers is built to distinguish the object from the background.
  2. Middle Layer (Base Ensembles): These weak classifiers are combined into "base ensembles" for each patch. The system then aggregates these patches using a weighted ensemble strategy. This creates a generative model that can search for the object even if some patches are obscured or deformed.
  3. Top Layer (Temporal Consistency): To prevent erratic tracking "jumps," an adaptive Kalman filter models the object's velocity and position. This provides a temporal "sanity check" for the visual detections.
The proposed hierarchical ensemble framework architecture.

Key Insights and Implementation

A central innovation of the HET framework is the use of randomly sampled, overlapping patches. Unlike rigid grid-based methods, this redundancy allows the tracker to remain locked on even if a significant portion of the object changes shape or is blocked.

Illustration of the overlapping patch strategy and the ensemble summation for local features.

To maintain real-time performance, the authors utilize a sparse random projection matrix (Compressive Sensing). This reduction in computational load allows the system to achieve 30 FPS on a standard i7 CPU, making it suitable for deployment on mobile robotic platforms.

Experimental Results

The framework was evaluated on the Online Object Tracking Benchmark (OTB) and a specialized Human Body Tracking subset. Key findings include:

  • Superior Accuracy: HET outperformed the base Compressive Tracking (CT) method by 24% in success rate and 38.3% in precision.
  • Robustness to Deformation: In tests targeting human movement, HET ranked at the top across several attributes, including occlusion (OCC), deformation (DEF), and fast motion (FM), outperforming 11 other state-of-the-art algorithms such as TLD, Struck, and MIL.
  • Real-Time Efficiency: The system consistently maintained 30 FPS, meeting the requirements for live robotic navigation without requiring GPU acceleration.

Conclusion

The Hierarchical Ensemble Tracking framework provides a robust and efficient solution for real-time object tracking in robotics. By nesting local classifiers within a global motion model, it effectively handles the complexities of dynamic environments. While it may not reach the absolute peak accuracy of the most resource-intensive deep learning models, its ability to deliver high performance on standard CPUs makes it a practical and powerful tool for the next generation of autonomous robots.

Summary updated

Abstract

Autonomous robots enjoy a wide popularity nowadays and have been applied in many applications, such as home security, entertainment, delivery, navigation and guidance. It is vital to robots to track objects accurately in these applications, so it is necessary to focus on tracking algorithms to improve the robustness and accuracy. In this paper, we propose a robust object tracking algorithm based on a hierarchical ensemble framework which can incorporate information including individual pixel features, local patches and holistic target models. The framework combines multiple ensemble models simultaneously instead of using a single ensemble model individually. A discriminative model which accounts for the matching degree of local patches is adopted via a bottom ensemble layer, and a generative model which exploits holistic templates is used to search for the object through the middle ensemble layer as well as an adaptive Kalman filter. We test the proposed tracker on challenging benchmark image sequences. Both qualitative and quantitative evaluations demonstrate that the proposed tracker performs superiorly against several state-of-the-art algorithms, especially when the appearance changes dramatically and the occlusions occur.
BibTeX
@inproceedings{wang2016robust,
  author = {Mengmeng Wang and Yong Liu},
  title = {{Robust Object Tracking with a Hierarchical Ensemble Framework}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2016},
  url = {https://openreview.net/forum?id=wq0cFl7cPKT},
}
Figure