Abstract
TRIC-track introduces a novel approach to visual object tracking by replacing traditional template matching with direct displacement regression. Utilizing Tracking by Regression with Incrementally Learned Cascades (TRIC), the system predicts 2D displacement vectors from sampled patches, aggregating them into a robust evidence map. By incorporating an implicit shape model through neighboring part features and a multi-temporal scale motion model, TRIC-track achieves state-of-the-art performance on benchmarks, particularly in handling non-rigid deformations and occlusions.
Introduction
Visual tracking is a cornerstone of computer vision, yet "generic" tracking—tracking an object without a pre-defined model—remains a significant challenge. Most existing trackers rely on template matching, where a patch of the object is used to search the next frame for the most similar area. This approach suffers from three major flaws: susceptibility to local optima in cluttered backgrounds, high computational costs due to exhaustive searching, and difficulty integrating spatial constraints for non-rigid objects.
The authors of TRIC-track propose a paradigm shift from "fitness-based" tracking (asking "is this the target?") to "displacement-based" tracking (asking "where is the target relative to this patch?").
Methodology
Direct Displacement Prediction and Cascaded Regression
The core of the system is Tracking by Regression with Incrementally Learned Cascades (TRIC). Instead of a binary match, the algorithm uses image features to predict a 2D displacement vector. Multiple sampled patches "vote" on a part's new location. Because correct predictions aggregate while errors tend to be random, the resulting evidence map is highly robust against noise and partial occlusions.
Borrowing from the Supervised Descent Method (SDM) typically used for facial landmarking, TRIC employs a sequence of regressors. The first level makes a coarse guess, while subsequent levels refine the prediction using increasingly localized features. While SDM is usually trained offline on massive datasets, this work demonstrates that it can be initialized on a single frame and successfully updated online.
Implicit Shape Model
To track complex objects, the regressor for a specific part (e.g., "Part A") considers features from both that part and its neighbors (e.g., "Part B" and "Part C"). This allows the system to learn spatial constraints implicitly. The "shape" of the object is baked into the regression weights rather than being defined by a rigid geometric formula, providing the benefits of a global shape model without the mathematical overhead of enforcing strict consistency.
Multi-Temporal Scale Motion Model (MTS)
To handle scenarios where an object is completely occluded, the system maintains motion models over different time scales. By analyzing short-term versus long-term trajectories, the tracker can predict where an object is likely to reappear after it has been hidden from view.
Experimental Results
The authors evaluated TRIC-track using the CVPR 2013 Benchmark, which includes 50 video sequences and 29 competing state-of-the-art trackers. TRIC-track ranked first overall at the time of publication.
- Precision: It achieved a precision score of 0.752, significantly outperforming then-leaders like Struck (0.656) and SCM (0.649).
- Success Rate: In terms of Area Under Curve (AUC) for overlap, it scored 0.530, surpassing the runner-up (SCM at 0.499).
- Ablation Study: The implicit shape model proved vital; removing it caused a 28% drop in performance on deformable objects, confirming that the learned spatial relationship between parts is essential for tracking non-rigid targets.
Conclusion and Limitations
TRIC-track reimagines tracking by ditching traditional search-based methods for direct regression. While highly effective, the system currently requires manual part initialization in the first frame. Additionally, because it is fundamentally a part-based point tracker, retro-fitting a bounding box for standard benchmarks can sometimes fail to capture the true extent of an object during extreme rotations. Future work may focus on automating part selection and improving the computational scaling for high-speed applications.
Summary updated