Faster and Simpler Siamese Network for Single Object Tracking

Shaokui Jiang, Baile Xu, Jian Zhao, Furao ShenarXiv 2021 · OpenReview

TL;DR

This paper introduces a faster and simpler Siamese network for single object tracking that achieves excellent accuracy even on low-end hardware. By incorporating Squeeze-and-Excitation (SE) blocks and a more efficient regression method, the proposed tracker significantly speeds up training and inference without sacrificing performance, making real-time tracking more accessible.

What Problem Does This Paper Address?

Single Object Tracking (SOT) is a crucial task in computer vision, with applications ranging from surveillance to autonomous driving. While deep learning, particularly Siamese networks, has significantly advanced SOT performance, these models have become increasingly deep and computationally expensive. This trend limits their deployment on resource-constrained devices and in real-time scenarios. The paper addresses this by aiming to strike a better balance between efficiency and accuracy, developing a tracker that is fast enough for low-end hardware while maintaining competitive performance.

Methods

The proposed approach builds upon the Siamese network architecture, which learns a target's appearance from a template and then searches for it in subsequent frames. Key innovations include:

  • Squeeze-and-Excitation (SE) Blocks: Integrated into the feature extractor, SE blocks act as an attention mechanism. They recalibrate channel-wise feature responses, allowing the network to focus on more informative features, leading to improved accuracy and faster convergence during training.
  • Efficient Regression: The paper proposes a novel, lightweight regression part. Instead of complex methods like region proposal networks (RPNs), it uses a 1x1 convolution kernel followed by a fully connected layer to predict the target's relative position. This significantly reduces computational overhead.
  • Shared Feature Extractor: Similar to other Siamese trackers, the network employs a shared feature extractor for both the template and search regions, promoting efficient learning of discriminative features.
  • Correlation Operation: The template features are convolved with the detection features to generate a response map, which is then fed into the regression part. The paper notes that their convolution result is smaller than in other Siamese networks, contributing to speed.

Datasets

The authors utilized two primary datasets for training:

  • GOT-10k: A large, high-diversity benchmark for generic object tracking, containing over 10,000 video clips and 1.5 million bounding boxes.
  • ImageNet-VID: A dataset providing image sequences with annotations for object positions, covering 30 basic-level categories.

For evaluation, the paper reports results on:

  • VOT2015, VOT2016, VOT2017: Standard benchmarks for evaluating visual object tracking performance, including metrics like accuracy, robustness, and Equivalent Filter Operations (EFO) for speed.
  • OTB100: A widely used dataset for evaluating tracking algorithms, featuring a diverse set of sequences.

Key Insights and Ideas

  • Attention for Efficiency: The integration of SE blocks demonstrates that attention mechanisms can not only improve accuracy but also contribute to faster training by guiding the network to learn more effectively.
  • Simplicity is Key for Speed: The paper highlights that complex regression mechanisms, while potentially offering marginal accuracy gains, can be a bottleneck for real-time performance. A simpler, efficient regression module can achieve comparable results with significantly less computational cost.
  • Training on Limited Data: The authors show that their network can be well-trained within a short period (5 epochs) using a relatively small amount of data (GOT-10k and ImageNet-VID), making it practical for researchers with limited resources.
  • Low-End Hardware Viability: The core message is that high-performance tracking doesn't necessarily require high-end GPUs. Their method is designed to be deployable on common, less powerful hardware.

Key Results

  • Speed: The proposed tracker achieves significantly faster running speeds compared to many state-of-the-art trackers, particularly on low-end hardware. For instance, on a GeForce 840M GPU, it achieves 44.4 fps, outperforming others like SiamRPN (19.5 fps) and SiamFC (12.6 fps).
  • Accuracy: Despite its simplicity and speed, the tracker maintains competitive accuracy. On VOT2015, it achieves 0.55 accuracy and a remarkable EFO of 52.37. On VOT2016, it shows 0.51 accuracy and 53.05 EFO. On VOT2017, it achieves 0.45 accuracy and 0.81 robustness.
  • Model Size: The network is notably small, making it suitable for deployment on mobile devices and embedded systems. Its size is comparable to or smaller than many other Siamese-based trackers.

Limitations and Future Work

  • Limitations: The paper acknowledges that while their tracker performs well, some state-of-the-art trackers might achieve slightly higher accuracy on specific benchmarks. The evaluation on VOT2017 also notes that some sequences might not be challenging enough for advanced trackers, potentially masking performance differences.
  • Future Work: The authors suggest that the fast processing speed of their tracker enables its integration with other tasks like object detection in real-time. This opens avenues for more complex, integrated vision systems. The paper also implies that further exploration of backbone architectures could yield even better results.

Summary updated

Abstract

Single object tracking (SOT) is currently one of the most important tasks in computer vision. With the development of the deep network and the release for a series of large scale datasets for single object tracking, siamese networks have been proposed and perform better than most of the traditional methods. However, recent siamese networks get deeper and slower to obtain better performance. Most of these methods could only meet the needs of real-time object tracking in ideal environments. In order to achieve a better balance between efficiency and accuracy, we propose a simpler siamese network for single object tracking, which runs fast in poor hardware configurations while remaining an excellent accuracy. We use a more efficient regression method to compute the location of the tracked object in a shorter time without losing much precision. For improving the accuracy and speeding up the training progress, we introduce the Squeeze-and-excitation (SE) network into the feature extractor. In this paper, we compare the proposed method with some state-of-the-art trackers and analysis their performances. Using our method, a siamese network could be trained with shorter time and less data. The fast processing speed enables combining object tracking with object detection or other tasks in real time.
BibTeX
@misc{jiang2021faster,
  author = {Shaokui Jiang and Baile Xu and Jian Zhao and Furao Shen},
  title = {{Faster and Simpler Siamese Network for Single Object Tracking}},
  year = {2021},
  eprint = {2105.03049},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2105.03049},
}
Figure