TL;DR
This paper introduces a novel object relocation visual tracking system for intelligent transportation that addresses the failure modes of existing Siamese network-based trackers. By incorporating a hierarchical arbiter to detect tracking failures and a corrector module utilizing a dynamic template set and the Hopcroft-Karp algorithm for object repositioning, the proposed method significantly improves tracking robustness and accuracy.
What Problem Does This Paper Address?
Autonomous driving and intelligent transportation systems heavily rely on accurate and robust visual tracking to understand the environment, localize vehicles, and plan paths. While Siamese network-based trackers have advanced the field by simplifying tracking into an image-matching problem, they still struggle with several critical issues:
- Limited Feature Representation: Lightweight neural networks can lead to insufficient feature extraction, making trackers vulnerable to distractors (e.g., similar objects, background clutter) and significant changes in viewing angles.
- Inability to Adapt to Object Variations: Trackers often fail when the target object undergoes substantial appearance changes.
- Failure to Reposition Lost Objects: Crucially, existing trackers lack an effective mechanism to detect when they've lost the target and subsequently re-localize it, leading to complete tracking failure.
This paper aims to tackle these limitations by developing a system that can not only track objects reliably but also recover from tracking failures.
Methods
The proposed system builds upon the SiamFC++ tracker and introduces two core novel components: an arbiter for failure detection and a corrector for object relocation.
- SiamFC++ Backbone: The system utilizes SiamFC++ as its foundation, which employs a Siamese network for feature extraction and a regression/classification head to generate a similarity map, identifying the object's most likely location.
- Hierarchical Arbiter: This module is designed to determine if the tracker has lost the object. It consists of two parts:
- Match Filter Arbiter: This component analyzes the histogram of Euclidean distances between the centers of candidate objects. A bipolar distribution in this histogram indicates the presence of distractors.
- Transfer Arbiter: This part further refines the decision by considering the relative positional changes of the object and potential distractors across frames. It uses K-means clustering and distance calculations (L1 and L2) to distinguish between the true object and a distractor.
- Corrector Module: When the arbiter detects a tracking failure, the corrector takes over to re-localize the object. It comprises:
- Dynamic Template Set: Instead of a single template, this module maintains a set of templates. It dynamically updates this set by backtracking through past frames to find images with significant appearance changes, thus improving the tracker's ability to handle variations.
- Assignment Algorithm (Hopcroft-Karp): To efficiently select the most relevant candidate objects from the similarity map, the Hopcroft-Karp algorithm is employed for bipartite graph matching. This helps identify potential "winners" that are similar to the object.
- Gradient Magnitude Similarity Deviation (GMSD): Finally, GMSD is used to precisely relocate the object by comparing the candidate "winners" with the dynamic template set. GMSD leverages gradient information to distinguish objects even with similar semantic information.
Datasets
The paper uses several standard datasets for training and evaluation:
- Training: OTB2015, VOT2018, LaSOT, and GOT-10k. The GOT-10k dataset was used specifically for training the tracker on its training subset.
- Evaluation: OTB2015, VOT2018, GOT-10k, and LaSOT.
Key Insights and Ideas
- The Importance of Failure Detection: The paper strongly emphasizes that simply improving tracking accuracy isn't enough. A robust system must also reliably detect when it has failed and then have a mechanism to recover.
- Histogram of Distances as a Distractor Detector: Using the distribution of distances between candidate object centers as a histogram is an elegant way to identify the presence of distractors, which often manifest as a bipolar distribution.
- Dynamic Template Sets for Robustness: Moving from a single template to a dynamic set that captures appearance variations is crucial for long-term tracking and handling object changes.
- Hopcroft-Karp for Efficient Candidate Selection: Leveraging a well-established algorithm like Hopcroft-Karp for bipartite graph matching significantly improves the efficiency of selecting candidate objects for relocation.
- GMSD for Precise Relocation: GMSD proves to be an effective metric for fine-grained object localization, especially when dealing with similar-looking objects or distractors.
Key Results
The proposed tracker demonstrates superior performance compared to state-of-the-art methods on several challenging benchmarks:
- OTB100: Achieved a success score of 0.727, outperforming other trackers.
- VOT-18: Achieved an accuracy score of 0.533, showing significant improvement over SiamFC++.
- LaSOT: Reached a success score of 0.575, indicating good performance in long-term tracking scenarios.
- GOT-10k: Achieved a success score of 0.612.
The arbiter module also showed promising results, with a high Tracking Invalid Accuracy Ratio (TIAR) across datasets, indicating its effectiveness in correctly identifying tracking failures.
Limitations and Future Work
- GOT-10k Performance: The paper notes that while the tracker performs well, the GOT-10k dataset's characteristics (small objects, significant viewpoint changes) present challenges that limit achieving even better performance.
- Arbiter Judgment Failures: The arbiter can still experience judgment failures in scenarios with extreme scale variations or highly similar interfering targets.
- Future Work: The authors plan to explore a self-supervised contrast learning method to construct an efficient online learning tracking corrector, aiming to further enhance tracking accuracy and real-time performance.
Summary updated