Aggregation Signature for Small Object Tracking

Chunlei Liu, Wenrui Ding, Jinyu Yang, Vittorio Murino, Baochang Zhang, Jungong Han, Guodong Guo

Abstract

Small object tracking is an increasingly important yet largely unexplored task in computer vision. Challenges arise from the vague appearance of small objects and their tendency to be lost due to camera shake. We propose a novel Aggregation Signature (AS) based on saliency to address sudden and large drifts. Our contributions include a new descriptor for distinctive features, a theoretical proof of its accuracy, and two new benchmarks: small90 and small112. Experimental results show our method outperforms state-of-the-art trackers by significant margins.

Introduction

In computer vision, tracking targets that occupy less than 1% of the image area presents unique difficulties. Standard algorithms often fail due to feature scarcity, where deep learning models lack sufficient pixels for semantic features, and sudden drift, where minor camera movements cause the target to jump significantly between frames. Existing saliency methods often struggle to suppress background clutter effectively for these "implicitly weak" objects.

Methodology: The Aggregation Signature

The core of our approach is the Aggregation Signature (AS), which leverages the frequency domain via the Quaternion Discrete Cosine Transform (QDCT). Unlike traditional RGB-based methods, the AS integrates four distinct channels: intensity, saturation, motion (temporal difference), and a target prior.

The four channels integrated into the Aggregation Signature framework.

Iterative Learning and Re-detection

The AS is not a static handcrafted feature but an online learning framework. It iteratively updates the signature to enhance target saliency while suppressing background noise. We mathematically prove that this iterative process increases the probability of matching the foreground object accurately.

The resulting Aggregation Signature Tracker (AST) employs a two-stage process:

  1. Drifting Detection: Triggered when the base tracker's confidence score falls below a threshold.
  2. Saliency Re-localization: The AS generates a saliency map to relocate the target and reset the base tracker.
The saliency map after the 4th iteration, showing effective background suppression.

Datasets and Evaluation

We validated the AS descriptor using the MSRA-B saliency benchmark and introduced two new datasets: small90 and small112. These benchmarks contain 90 and 112 sequences specifically designed to test tracking under low resolution and camera shake. We also evaluated performance on the UAV123 and UAV20L drone datasets.

Results

Our AST framework significantly improves tracking performance:

  • Accuracy: On the small90 dataset, AST-MDNet achieved 86.6% precision, a 7.1% improvement over the base MDNet.
  • Robustness: The method outperformed heavyweights like SiamRPN and ECO in scenarios involving high motion blur and camera shake.
  • Efficiency: When integrated with the KCF tracker, the system reached 120.88 FPS, enabling real-time application.

Conclusion

The Aggregation Signature provides a robust mechanism for re-detecting small objects during drift. While its primary value lies in recovery from loss, its ability to suppress background clutter in the frequency domain is a powerful tool for small-scale computer vision. Future work will explore applying this saliency-based learning to image retrieval and classification tasks.

Summary updated

Abstract

Small object tracking becomes an increasingly important task, which however has been largely unexplored in computer vision. The great challenges stem from the facts that: 1) small objects show extreme vague and variable appearances, and 2) they tend to be lost easier as compared to normal-sized ones due to the shaking of lens. In this paper, we propose a novel aggregation signature suitable for small object tracking, especially aiming for the challenge of sudden and large drift. We make three-fold contributions in this work. First, technically, we propose a new descriptor, named aggregation signature, based on saliency, able to represent highly distinctive features for small objects. Second, theoretically, we prove that the proposed signature matches the foreground object more accurately with a high probability. Third, experimentally, the aggregation signature achieves a high performance on multiple datasets, outperforming the state-of-the-art methods by large margins. Moreover, we contribute with two newly collected benchmark datasets, i.e., small90 and small112, for visually small object tracking. The datasets will be available in https://github.com/bczhangbczhang/.
BibTeX
@article{liu2020aggregation,
  author = {Chunlei Liu and Wenrui Ding and Jinyu Yang and Vittorio Murino and Baochang Zhang and Jungong Han and Guodong Guo},
  title = {{Aggregation Signature for Small Object Tracking}},
  journal = {IEEE Trans. Image Processing2020},
  year = {2020},
  url = {https://openreview.net/forum?id=MhxZPzoZszVs},
}
Figure