Abstract
Visual tracking using deep learning often faces a trade-off between high accuracy and real-time performance. This paper introduces the Multi-Scale Domain Adaptation Tracker (MSDAT), which leverages channel reduction and a "lazy feed-forward" strategy to achieve state-of-the-art precision at approximately 35 FPS. By adapting pre-trained classification features for instance-specific tracking, MSDAT bridges the gap between general visual recognition and robust object following.
Introduction
Visual tracking—the task of following a specific object across a video sequence—has seen a massive performance boost thanks to Deep Convolutional Neural Networks (CNNs). However, this accuracy comes at a high computational cost. Most high-performing deep trackers are slow (often 1–10 FPS), making them impractical for real-world use in robotics or surveillance. Conversely, existing "real-time" trackers often sacrifice significant precision to maintain speed.
Furthermore, most deep trackers use features pre-trained for image classification (e.g., distinguishing a "car" from a "dog"). Tracking requires a different focus: distinguishing a specific instance of an object from its immediate background, even if the background contains other objects of the same category. This "domain gap" necessitates specialized adaptation techniques.
Methodology
The authors introduce MSDAT, which builds upon the Hierarchical Convolutional Features (HCF) framework but introduces several clever optimizations to achieve real-time efficiency without losing robustness.
Convolutional Channel Reduction and Multi-Scale Adaptation
The core of MSDAT is a specialized tracking branch "grafted" onto the feature layers of a pre-trained VGG-19 network. This branch consists of convolutional layers that reduce the number of feature channels by a factor of 8 (e.g., from 256 channels down to 32). This domain adaptation strips away redundant classification data and focuses on features essential for identifying individual objects. To handle objects of varying sizes, the adaptation layer uses multiple filter sizes (3x3 and 5x5), inspired by Inception architectures, which are concatenated to provide a richer representation.
Lazy Feed-Forward Strategy
In standard tracking, the CNN typically runs twice per frame: once for prediction and once for model updates. MSDAT employs a "Lazy Feed-Forward" strategy based on the observation that objects usually move very little between consecutive frames. By cropping a search window slightly larger than the target, the system can reuse feature maps for the update step if the object remains within that window, resulting in a nearly 50% speed gain.
KCF Integration
The reduced, multi-scale features are fed into Kernelized Correlation Filters (KCF). KCF is a highly efficient classical tracking method that benefits immensely from the lower dimensionality of the adapted features, as its computational complexity is heavily tied to the number of input channels.
Experimental Results
MSDAT was evaluated on the industry-standard OTB-50 and OTB-100 (Object Tracking Benchmark) datasets. The base VGG-19 network was pre-trained on the ILSVRC (ImageNet) dataset for general feature extraction.
Performance and Speed
- Speed: MSDAT runs at 34.8 FPS, meeting the threshold for real-time applications. This is roughly 3x faster than its prototype (HCF).
- Accuracy: On the OTB-100 benchmark, MSDAT achieved a Distance Precision (DP) rate of 83.0% and an Area Under Curve (AUC) of 0.567.
- Competitive Edge: While the "MD-Net" tracker is more accurate (90.9% DP), it runs at only 1 FPS. Compared to other real-time trackers like GOTURN (165 FPS but 56.39% DP) and SiamFC (58 FPS and 75.2% DP), MSDAT provides a superior balance of high precision and usable speed.
Conclusion
MSDAT successfully addresses the accuracy-speed dilemma in deep visual tracking. By utilizing channel reduction to bridge the domain gap between classification and tracking, and implementing a lazy feed-forward strategy to exploit temporal redundancy, the tracker achieves state-of-the-art results at real-time speeds. Future work will explore online updating of the neural network weights to further boost robustness without compromising the real-time efficiency established in this study.
Summary updated