Abstract
Visual object tracking is a fundamental computer vision task complicated by the competing demands of handling partial occlusion and significant appearance variations. This paper proposes a collaborative tracking framework that integrates a shallow, part-based generative model with a deep, holistic discriminative model. By employing a block-based occlusion mask and a Deep Belief Network (DBN) for feature extraction and online adaptation, the tracker achieves state-of-the-art robustness. Evaluations on standard benchmarks demonstrate that this collaborative approach effectively balances local detail with global context, outperforming several existing methods in challenging environments.
Introduction
The primary difficulty in visual tracking lies in the inherent tension between holistic and part-based modeling. Holistic models, which consider the object as a single entity, excel at distinguishing targets from cluttered backgrounds and adapting to global appearance changes. However, they are highly sensitive to partial occlusions. Conversely, part-based (local) models are inherently robust to occlusion—as unhidden parts can still guide the tracker—but often lack the contextual information necessary to prevent drifting toward similar-looking background elements.
This research addresses this gap by asking whether a collaborative system can leverage the strengths of both approaches: shallow, local features to survive occlusion and deep, holistic features to manage appearance changes.
Methodology
The proposed system operates within a particle filter framework, where candidate target locations are generated and scored based on two collaborative components.
1. Shallow Generative Model for Occlusion Handling
To mitigate the impact of occlusions, the algorithm divides candidate bounding boxes into a 4x4 grid of blocks. It utilizes Principal Component Analysis (PCA) to calculate similarity scores for each block relative to the known target. A key innovation is the implementation of an occlusion mask. If a specific block's similarity score falls below a predefined threshold, the system identifies it as occluded and zeroes out its contribution. This prevents the "corrupting" influence of the occluding object from affecting the overall tracking score.
2. Deep Discriminative Model for Appearance Adaptation
For handling complex lighting, rotation, and shape changes, the researchers employ a Deep Belief Network (DBN). Unlike the part-based model, the DBN processes the entire candidate box. The network is pre-trained offline on a large dataset of image patches to learn discriminative features that separate foreground objects from backgrounds. During the tracking process, if the network's confidence in the target's identity drops, it triggers an online fine-tuning mechanism, updating its weights to incorporate the target's current appearance.
3. Collaborative Scoring
The final selection of the target is determined by the product of the generative and discriminative scores. This multiplicative approach acts as a rigorous filter, requiring a candidate to match both the visible local parts of the target and the deep network's holistic representation of the foreground.
Experimental Evaluation
The DBN was pre-trained on a custom dataset of 100,000 grayscale image patches (50/50 split of foreground and background). The resulting tracker was evaluated against 13 state-of-the-art algorithms on 25 challenging sequences from the Wu et al. (2013) benchmark. These sequences include scenarios with heavy occlusion, motion blur, and extreme illumination changes.
Key Results and Insights
- Superior Accuracy and Robustness: The collaborative tracker consistently ranked in the top three across all test sequences. It showed particular strength in sequences involving heavy occlusion (e.g., faceocc1) and severe illumination shifts (e.g., carDark).
- Classification vs. Reconstruction: A significant finding was that training the deep network for classification (foreground vs. background) yielded more robust features than training it as a simple autoencoder for image reconstruction.
- Efficiency: Despite the complexity of the deep model, the algorithm achieves a processing speed of 7 frames per second (FPS) on a standard CPU. This is made possible because the deep network only requires a single forward pass for evaluation per particle.
Conclusion
By combining a mathematically grounded occlusion mask with the high-level feature extraction of a Deep Belief Network, this collaborative model successfully bridges the gap between local and holistic tracking. While the current implementation's speed may limit its use in ultra-high-speed real-time applications, the framework provides a robust foundation for future trackers. Potential improvements include the use of more efficient convolutional architectures and the development of dynamic thresholds to further automate the tracking process in varying environments.
Summary updated