Object Relocation Visual Tracking Based on Histogram Filter and Siamese Network in Intelligent Transportation

Jianlong Zhang, Yifan Liu, Qiao Li, Ci He, Bin Wang, Tianhong WangSensors 2022 · DOI

TL;DR

This paper introduces a novel object relocation visual tracking system for intelligent transportation that addresses the failure modes of existing Siamese network-based trackers. By incorporating a hierarchical arbiter to detect tracking failures and a corrector module utilizing a dynamic template set and the Hopcroft-Karp algorithm for object repositioning, the proposed method significantly improves tracking robustness and accuracy.

What Problem Does This Paper Address?

Autonomous driving and intelligent transportation systems heavily rely on accurate and robust visual tracking to understand the environment, localize vehicles, and plan paths. While Siamese network-based trackers have advanced the field by simplifying tracking into an image-matching problem, they still struggle with several critical issues:

  • Limited Feature Representation: Lightweight neural networks can lead to insufficient feature extraction, making trackers vulnerable to distractors (e.g., similar objects, background clutter) and significant changes in viewing angles.
  • Inability to Adapt to Object Variations: Trackers often fail when the target object undergoes substantial appearance changes.
  • Failure to Reposition Lost Objects: Crucially, existing trackers lack an effective mechanism to detect when they've lost the target and subsequently re-localize it, leading to complete tracking failure.

This paper aims to tackle these limitations by developing a system that can not only track objects reliably but also recover from tracking failures.

Methods

The proposed system builds upon the SiamFC++ tracker and introduces two core novel components: an arbiter for failure detection and a corrector for object relocation.

  • SiamFC++ Backbone: The system utilizes SiamFC++ as its foundation, which employs a Siamese network for feature extraction and a regression/classification head to generate a similarity map, identifying the object's most likely location.
  • Hierarchical Arbiter: This module is designed to determine if the tracker has lost the object. It consists of two parts:
  • Match Filter Arbiter: This component analyzes the histogram of Euclidean distances between the centers of candidate objects. A bipolar distribution in this histogram indicates the presence of distractors.
  • Transfer Arbiter: This part further refines the decision by considering the relative positional changes of the object and potential distractors across frames. It uses K-means clustering and distance calculations (L1 and L2) to distinguish between the true object and a distractor.
  • Corrector Module: When the arbiter detects a tracking failure, the corrector takes over to re-localize the object. It comprises:
  • Dynamic Template Set: Instead of a single template, this module maintains a set of templates. It dynamically updates this set by backtracking through past frames to find images with significant appearance changes, thus improving the tracker's ability to handle variations.
  • Assignment Algorithm (Hopcroft-Karp): To efficiently select the most relevant candidate objects from the similarity map, the Hopcroft-Karp algorithm is employed for bipartite graph matching. This helps identify potential "winners" that are similar to the object.
  • Gradient Magnitude Similarity Deviation (GMSD): Finally, GMSD is used to precisely relocate the object by comparing the candidate "winners" with the dynamic template set. GMSD leverages gradient information to distinguish objects even with similar semantic information.

Datasets

The paper uses several standard datasets for training and evaluation:

  • Training: OTB2015, VOT2018, LaSOT, and GOT-10k. The GOT-10k dataset was used specifically for training the tracker on its training subset.
  • Evaluation: OTB2015, VOT2018, GOT-10k, and LaSOT.

Key Insights and Ideas

  • The Importance of Failure Detection: The paper strongly emphasizes that simply improving tracking accuracy isn't enough. A robust system must also reliably detect when it has failed and then have a mechanism to recover.
  • Histogram of Distances as a Distractor Detector: Using the distribution of distances between candidate object centers as a histogram is an elegant way to identify the presence of distractors, which often manifest as a bipolar distribution.
  • Dynamic Template Sets for Robustness: Moving from a single template to a dynamic set that captures appearance variations is crucial for long-term tracking and handling object changes.
  • Hopcroft-Karp for Efficient Candidate Selection: Leveraging a well-established algorithm like Hopcroft-Karp for bipartite graph matching significantly improves the efficiency of selecting candidate objects for relocation.
  • GMSD for Precise Relocation: GMSD proves to be an effective metric for fine-grained object localization, especially when dealing with similar-looking objects or distractors.

Key Results

The proposed tracker demonstrates superior performance compared to state-of-the-art methods on several challenging benchmarks:

  • OTB100: Achieved a success score of 0.727, outperforming other trackers.
  • VOT-18: Achieved an accuracy score of 0.533, showing significant improvement over SiamFC++.
  • LaSOT: Reached a success score of 0.575, indicating good performance in long-term tracking scenarios.
  • GOT-10k: Achieved a success score of 0.612.

The arbiter module also showed promising results, with a high Tracking Invalid Accuracy Ratio (TIAR) across datasets, indicating its effectiveness in correctly identifying tracking failures.

Limitations and Future Work

  • GOT-10k Performance: The paper notes that while the tracker performs well, the GOT-10k dataset's characteristics (small objects, significant viewpoint changes) present challenges that limit achieving even better performance.
  • Arbiter Judgment Failures: The arbiter can still experience judgment failures in scenarios with extreme scale variations or highly similar interfering targets.
  • Future Work: The authors plan to explore a self-supervised contrast learning method to construct an efficient online learning tracking corrector, aiming to further enhance tracking accuracy and real-time performance.

Summary updated

Abstract

Target detection and tracking algorithms are one of the key technologies in the field of autonomous driving in intelligent transportation, providing important sensing capabilities for vehicle localization and path planning. Siamese network-based trackers formulate the visual tracking mission as an image-matching process by regression and classification branches, which simplifies the network structure and improves the tracking accuracy. However, there remain many problems, as described below. (1) The lightweight neural networks decrease the feature representation ability. It is easy for the tracker to fail under the disturbing distractors (e.g., deformation and similar objects) or large changes in the viewing angle. (2) The tracker cannot adapt to variations of the object. (3) The tracker cannot reposition the object that has failed to track. To address these issues, we first propose a novel match filter arbiter based on the Euclidean distance histogram between the centers of multiple candidate objects to automatically determine whether the tracker fails. Secondly, the Hopcroft-Karp algorithm is introduced to select the winners from the dynamic template set through the backtracking process, and object relocation is achieved by comparing the Gradient Magnitude Similarity Deviation between the template and the winners. The experiments show that our method obtains better performance on several tracking benchmarks, i.e., OTB100, VOT2018, GOT-10k, and LaSOT, compared with state-of-the-art methods.
BibTeX
@inproceedings{zhang2022object,
  author = {Jianlong Zhang and Yifan Liu and Qiao Li and Ci He and Bin Wang and Tianhong Wang},
  title = {{Object Relocation Visual Tracking Based on Histogram Filter and Siamese Network in Intelligent Transportation}},
  booktitle = {Sensors},
  year = {2022},
  doi = {10.3390/s22228591},
}
Figure