Abstract: Visual object tracking has seen significant advances through the tracking-by-detection paradigm, particularly with the integration of deep convolutional neural networks. However, standard trackers often rely solely on correlation or feature selection, frequently neglecting physical motion constraints. This paper investigates the integration of rotation adaptiveness and motion consistency into high-speed tracking frameworks. By capturing physical constraints, the proposed approach improves accuracy, robustness, and rotation adaptiveness across challenging visual sequences.
1. Introduction
In modern computer vision, "tracking by detection" using architectures like Siamese Fully Convolutional Networks (SiameseFC) and Correlation Filter Networks (CFnet) has become the standard due to their high processing speeds. Despite their efficiency, these trackers possess a significant "blind spot": they typically assume targets remain axis-aligned. When an object undergoes rotation, severe deformation, or erratic movement, these trackers often drift, failing to maintain an accurate bounding box.
Furthermore, most current methods ignore the physical constraints of motion—the principle that objects generally follow smooth, predictable trajectories. This paper addresses these gaps by introducing a framework for Rotation Adaptive Tracking and Motion Consistency, enforcing smoothness in both direction and distance to prevent the tracker from making physically impossible "jumps" caused by background noise.
2. Proposed Methodology
The authors propose a generic approach to enhance existing state-of-the-art trackers, specifically creating "DSR" (Displacement, Scale, and Rotation) versions of SiameseFC and CFnet.
2.1 Base Architecture
The foundation of the approach relies on the Siamese network paradigm, which compares a target exemplar ($z$) against a search image ($x$) to generate a response map. The proposed DSR framework modifies this pipeline to handle geometric deformations.
2.2 Rotation Invariance (RI)
To handle spinning objects, the Siamese DSR tracker extracts features from multiple rotated versions of the target template. It utilizes a Gaussian-weighted average of the nearest neighbor response maps to identify the optimal orientation. For CFnet DSR, which updates every frame, the model applies forward and backward rotations after each update to account for gradual orientation shifts without requiring a massive bank of pre-rotated templates.
2.3 Motion and Scale Consistency
Instead of simply selecting the highest score in a response map—a method prone to noise—the algorithm enforces consistency across frames:
- Displacement Consistency: The tracker maintains a rolling average of the motion vector's angle and distance. By assigning a small weight (1%) to the previous motion vector, the system acts as a regularizer, ensuring the trajectory remains physically plausible.
- Scale Consistency: Rather than picking a single winning scale from a pyramid, the system computes a Gaussian weighted average of response maps across different scales, stabilizing the bounding box and reducing jitter.
3. Experimental Results
The DSR-enhanced trackers were evaluated on major industry benchmarks, including OTB (Object Tracking Benchmark) and VOT (Visual Object Tracking) 2016, focusing on sequences with extreme rotations like glove and fish1.
- VOT 2016 Performance: The Siamese DSR achieved a 15.7% improvement in robustness rank and a 14.3% improvement in accuracy rank over the baseline SiameseFC.
- OTB Performance: On OTB100, the success rate (AUC) improved by 4.6%, while precision increased by 6.75%.
- Optimization: Empirical testing revealed that checking for rotations at 8-degree intervals provided the optimal balance between tracking accuracy and computational overhead.
4. Conclusion
By treating physics as a regularizer, the proposed framework significantly reduces tracker drift in the presence of background clutter and geometric changes. While current evaluation toolkits often penalize rotation estimation due to their reliance on axis-aligned bounding boxes, the performance gains observed here demonstrate the clear value of rotation adaptiveness. Future work will explore integrating these motion consistency techniques into deeper architectures like MDNet and TCNN to further refine feature extraction in rotation-invariant branches.
Summary updated