SVTN: Siamese Visual Tracking Networks With Spatially Constrained Correlation Filter and Saliency Prior Context Model

Bo Huang, Tingfa Xu, Shenwang Jiang, Yu Bai, Yiwen Chen

Visual tracking—the task of following a specific object across a video—faces several persistent hurdles: target occlusion, scale changes, and background distractors. While Siamese networks have become the gold standard for feature extraction due to their speed and offline learning capabilities, their decision-making mechanism (often simple cross-correlation) is frequently too shallow to handle complex environments.

Furthermore, traditional Correlation Filters (CFs) suffer from the "boundary effect," where the periodic assumption of the Fast Fourier Transform (FFT) introduces artificial noise at the edges of the search window. Finally, most trackers struggle with "model poisoning"—updating the tracker's internal model while the target is occluded, which leads to permanent tracking failure. To address these challenges, the authors propose SVTN (Siamese Visual Tracking Network), which integrates a Spatially Constrained Correlation Filter (SCCF) and a Saliency Prior Context (SPC) model.

Methodology

The SVTN architecture combines deep learning feature extraction with classical constrained optimization to create a more robust tracking pipeline.

The overall architecture of the SVTN framework, illustrating the flow from Siamese feature extraction to the final bounding box estimation via SCCF and SPC modules.

Siamese Feature Extraction and SCCF

The network uses a pre-trained Siamese backbone to extract deep, discriminative features from both the target template and the search region. To eliminate boundary effects, the authors introduce a Spatially Constrained Correlation Filter (SCCF). This involves a penalizing matrix that assigns higher weights to the background region during training, forcing the filter to focus on the target. This constrained optimization problem is solved efficiently using the Alternating Direction Method of Multipliers (ADMM), allowing for spatial constraints while maintaining the speed of frequency-domain computations.

Saliency Prior Context (SPC) Model

The SPC model treats the Siamese network's output as a "prior probability." By applying a spatio-temporal relationship and a Kronecker product between the saliency map and the feature map, the tracker can better distinguish the target from similar-looking distractors in the background. This acts as a secondary verification step, effectively "muting" background areas that look like the target but are spatially inconsistent.

Visualization of the Saliency Prior Context (SPC) model, showing how the saliency map G(x) is used to generate a masked target representation M(x).

Occlusion-Aware Update

Instead of updating every frame, SVTN uses a Gaussian Mixture Model (GMM) to reconstruct the target. If the reconstruction error exceeds a certain threshold, the system identifies the target as occluded and skips the update. This "selective memory" prevents the model from learning unreliable data during periods of occlusion.

Experimental Results

SVTN was evaluated on three major industry-standard benchmarks: OTB-2013, OTB-2015, and VOT-2016. The model demonstrated superior performance over several state-of-the-art trackers, including BACF, SRDCF, and Staple.

  • OTB-2015: Achieved a precision score of 0.826 and a success score of 0.626, significantly outperforming the baseline CFNet2s (0.748 and 0.568).
  • VOT-2016: SVTN ranked 1st in Accuracy (0.546) and Expected Average Overlap (0.346) among the compared trackers.
  • Robustness to Scale: In sequences involving significant scale changes, SVTN provided a 10% gain over the SAMF tracker, a previous milestone in scale handling.

Conclusion and Future Work

SVTN successfully bridges the gap between deep Siamese features and classical constrained optimization. However, this accuracy comes with increased computational complexity due to the iterative nature of the ADMM solver and the GMM reconstruction process. Additionally, the tracker performs less effectively in Low Resolution (LR) sequences where discriminative information is scarce. Future work may involve optimizing ADMM iterations for real-time performance or integrating multi-scale feature fusion to better handle low-resolution targets.

Summary updated

BibTeX
@article{huang2019svtn,
  author = {Bo Huang and Tingfa Xu and Shenwang Jiang and Yu Bai and Yiwen Chen},
  title = {{SVTN: Siamese Visual Tracking Networks With Spatially Constrained Correlation Filter and Saliency Prior Context Model}},
  journal = {IEEE Access},
  year = {2019},
  doi = {10.1109/access.2019.2945846},
}
Figure