Abstract
Modern visual tracking often relies on parallel refinement of object proposals, which can lead to feature misalignment and vulnerability to distractors. This paper presents CRACT, a tracker featuring a novel Cascaded Regression-Align-Classification (CRAC) module. By sequentially performing box regression, feature alignment, and classification, CRACT ensures that the classifier operates on accurate, noise-free features. Coupled with a multi-scale Pyramid RoIAlign and a dual-component classification system (offline identification and online discrimination), CRACT achieves state-of-the-art performance across seven major benchmarks, including OTB-2015 and UAV123, while maintaining real-time performance at 28 FPS.
The Misalignment Problem in Visual Tracking
In typical Siamese-based tracking architectures, a Region Proposal Network (RPN) generates initial target candidates. These proposals are then refined through two parallel branches: box regression and box classification.
The authors identify two critical flaws in this parallel approach:
- Feature Contamination: Since classification occurs simultaneously with regression, the features for classification are extracted from the initial, often inaccurate, proposal locations. This misalignment introduces background noise, degrading classification accuracy.
- Distractor Vulnerability: Many trackers rely solely on offline template matching, making them susceptible to "distractors"—background objects that resemble the target.
The CRACT Framework
To address these issues, CRACT introduces the Cascaded Regression-Align-Classification (CRAC) module. Instead of parallel branches, CRAC employs a three-step sequential pipeline.
Sequential Refinement (CRAC)
- Box Regression: The module first predicts offsets to adjust the initial proposal's scale and position.
- Feature Alignment: This is the critical "bridge." The regression offsets are used to "re-align" the features. By re-sampling features from the adjusted bounding box, the model ensures the classifier receives the most accurate representation of the target.
- Dual-Component Classification: Classification is split into an Identifier and a Discriminator. The Identifier uses a relation network for stable offline template matching. The Discriminator is a lightweight network updated online to learn the current background and suppress distractors.
Pyramid RoIAlign (PRoIAlign)
Standard RoIAlign pools features to a single fixed size. CRACT proposes Pyramid RoIAlign, which pools features at three scales (6x6, 3x3, and 1x1). Concatenating these scales allows the model to capture both fine-grained local details and broader global context, improving robustness against large deformations.
Experimental Results
CRACT was evaluated on seven diverse benchmarks, including OTB-2015, UAV123, NfS, VOT-2018, TrackingNet, GOT-10k, and LaSOT.
- State-of-the-Art Performance: On OTB-2015, CRACT achieved a Success (SUC) score of 0.726, a 1.4% improvement over SiamAttn. It also led on UAV123 (0.664 SUC) and GOT-10k (0.620 AO).
- Ablation Studies: The importance of the cascaded architecture was confirmed by removing the alignment step, which resulted in a performance drop from 0.726 to 0.713 SUC on OTB-2015.
- Efficiency: Despite its complexity, CRACT runs at 28 FPS on a single GTX 1080 GPU, meeting real-time requirements.
Conclusion
CRACT demonstrates that the sequence of refinement operations is as important as the operations themselves. By bridging regression and classification with a feature alignment step, CRACT significantly improves tracking robustness. Future work may involve integrating pixel-wise mask segmentation into this cascaded framework to further enhance accuracy for complex object shapes.
Summary updated