The State-of-the-Art in Handling Occlusions for Visual Object Tracking

Kourosh Meshgi, Shin IshiiIEICE Trans. Information and Systems 2015 · DOI

Introduction

Visual object tracking is a fundamental task in computer vision, yet it remains highly susceptible to failure in dynamic environments. One of the most significant challenges is occlusion, where the target object is partially or entirely obscured by other objects or background elements. This survey provides a comprehensive analysis of how modern trackers address occlusions, proposing a systematic framework to categorize these scenarios and evaluating the effectiveness of various algorithmic strategies.

The Occlusion Problem

While tracking performance has improved significantly, occlusion often leads to "model drift." This occurs when a tracker erroneously updates its appearance model using features from the occluder rather than the target. To move beyond qualitative descriptions of "robustness," this paper introduces a rigorous taxonomy to classify occlusions based on three binary dimensions:

  1. Extent: Partial vs. Full occlusion.
  2. Duration: Short-term (temporal) vs. Long-term (persistent).
  3. Complexity: Simple (stable target appearance) vs. Complex (changes in orientation, scale, or motion during occlusion).
Examples of real-world occlusion challenges, including large occluders like trains and complex motion during crashes.

Methodological Strategies

The literature reveals several distinct approaches to mitigating occlusion-related failures:

  • Robust Representations: Utilizing local patches or superpixels allows trackers to maintain a lock on visible portions of an object even when others are hidden.
  • Motion Modeling: Predictive frameworks like Kalman or Particle filters estimate trajectories during periods of invisibility.
  • Occlusion Reasoning: Explicit depth ordering helps the system understand spatial relationships between targets and occluders.
  • Mode Switching: Implementing "occlusion detection" triggers that pause model updates prevents the tracker from learning the occluder's appearance.

A quantitative analysis of existing literature shows a heavy concentration on simpler occlusion types. Most research focuses on Partial-Temporal-Simple (PTS) scenarios, while Full-Long-Complex (FLC) cases remain largely unaddressed in the surveyed literature.

Distribution of tracking literature across the proposed three-dimensional occlusion taxonomy.

Key Insights and Performance

The survey identifies several critical findings:

  • The 30% Threshold: Partial occlusion up to 30% is largely considered a solved problem for modern trackers. The primary frontier lies in handling full and persistent occlusions.
  • The Update Trap: Successful trackers must know when not to learn. Pausing updates during detected occlusions is vital for preventing drift.
  • Parts vs. Wholes: Holistic models are fragile; sparse, part-based representations offer "graceful degradation" and higher resilience.
  • Contextual Support: Leveraging "supporter" objects in the target's vicinity can maintain tracking even when the target is 100% occluded.

Benchmarking and Future Directions

The paper reviews standard benchmarks (TB-50/100, ALOV++) and specialized datasets like SimOcc (simulated variables) and Princeton RGBD (depth-aided). Despite progress, several gaps remain:

  • Lack of Formalization: Most occlusion reasoning remains ad-hoc rather than mathematically formalized.
  • The "Birth and Death" Problem: Managing objects entering or leaving the scene mid-sequence is still a challenge.
  • Underutilization of Depth: While RGB-D data simplifies occlusion reasoning, 2D video remains the dominant research focus.

Future research is expected to move toward the fusion of feature detectors with trackers and more sophisticated foreground segmentation to achieve true robustness in complex environments.

Summary updated

BibTeX
@article{meshgi2015state,
  author = {Kourosh Meshgi and Shin Ishii},
  title = {{The State-of-the-Art in Handling Occlusions for Visual Object Tracking}},
  journal = {IEICE Trans. Information and Systems},
  year = {2015},
  doi = {10.1587/transinf.2014edr0002},
}
Figure