Abstract
Visual object tracking remains a challenging task due to non-rigid deformations, rotations, and lighting changes. This paper proposes a novel framework to learn hierarchical features specifically for tracking. By combining offline learning on auxiliary video sequences with a temporal slowness constraint and online domain adaptation for specific targets, the model achieves state-of-the-art performance. The learned features are robust to complex motion transformations and appearance shifts, significantly improving tracking accuracy and overlap rates.
Introduction
Visual object tracking is often hindered by the constant evolution of a target's appearance. Traditional methods rely on hand-crafted features like HOG or features learned from static image datasets. However, these fail to capture temporal invariance—the principle that an object’s features should transition smoothly over time. This paper addresses this gap by introducing a two-layer Convolutional Neural Network (CNN) that learns motion-robust features offline and adapts them to specific objects online.
Methodology: Offline Learning and Online Adaptation
The core of the proposed approach is a two-stage learning process designed to separate generic motion patterns from target-specific appearance details.
Offline Learning with Temporal Slowness
The network is first trained on auxiliary video sequences. To ensure the features are robust to motion, the authors employ a temporal slowness constraint. This constraint penalizes large fluctuations in feature representations between consecutive frames. Consequently, the first layer learns to handle simple translations, while the second layer captures more complex transformations such as non-rigid warping and out-of-plane rotations.
Online Domain Adaptation
When a tracking task begins, the generic features are tuned using a domain adaptation module. This module balances the preservation of robust pre-learned features with the incorporation of the specific target's appearance from the initial video frames. This adaptation occurs in both layers of the CNN.
As seen in the figure above, the adaptation process makes the filters visually more relevant to the target. For instance, in face tracking, the second-layer filters evolve to represent semantic, face-like parts.
Experimental Results
The method was evaluated on 15 challenging video sequences featuring complex motion and 8 standard benchmark sequences. The learned features were integrated into the Adaptive Structural Local sparse Appearance (ASLA) model.
Quantitative Performance
The proposed tracker significantly outperformed baselines. On the 15 challenging sequences, it achieved an average overlap rate of 63%, compared to just 36% for the raw-pixel baseline (ASLA_RAW).
The table above highlights that the proposed method ("Ours") consistently achieves the highest overlap rates across nearly all test sequences. Furthermore, the average center location error was reduced to 12.2 pixels, a massive improvement over the HOG-based tracking error of 89.3 pixels.
Discussion and Conclusion
The primary innovation of this work is the successful separation of "how things move" (learned offline) from "what a specific object looks like" (adapted online). While the current implementation faces computational challenges—running at approximately 0.8 frames per second—the robustness of the hierarchical features is clear. Future work focusing on GPU acceleration and parallel computing could bring this high-accuracy approach closer to real-time performance.
Reference: Wang, L., Liu, T., Wang, G., Chan, K. L., & Yang, Q. (2015). Video Tracking Using Learned Hierarchical Features. arXiv:1511.07940v1.
Summary updated