Integrating Boundary and Center Correlation Filters for Visual Tracking with Aspect Ratio Variation

Feng Li, Yingjie Yao, Peihua Li, David Zhang, Wangmeng Zuo, Ming-Hsuan Yang

Abstract

Visual tracking often fails when an object changes its proportions, such as a car turning or a person bending. This paper introduces IBCCF, a tracking framework that integrates a standard 2D center correlation filter with a family of four 1D boundary filters to independently track the target's edges. By enforcing a "near-orthogonality" constraint between these filters, the model achieves state-of-the-art robustness against aspect ratio variations.

Introduction

Most modern visual trackers rely on Correlation Filters (CF) because they are incredibly fast, leveraging the Fast Fourier Transform (FFT) to operate in the frequency domain. However, standard CF trackers have a major weakness: they typically assume the target's aspect ratio remains fixed.

While some "scale-adaptive" trackers can handle a target getting larger or smaller (isometric scaling), they struggle when the target deforms or rotates in a way that changes its height-to-width ratio. For example, a side-view of a car has a very different aspect ratio than a front-view. If a tracker cannot adjust its bounding box proportions, it eventually loses the target or includes too much background noise, leading to drift and failure.

Visual comparison of IBCCF against baseline trackers (HCF, sHCF) in sequences with significant aspect ratio changes.

Methodology: The IBCCF Framework

The authors propose the Integrating Boundary and Center Correlation Filters (IBCCF) model. The core innovation lies in how it rethinks the geometry of the target by decomposing the tracking task into center localization and boundary estimation.

1D Boundary Correlation Filters (BCFs)

Instead of relying solely on a 2D filter for the object's center, the authors introduce four 1D filters dedicated to localizing the left, right, top, and bottom boundaries. By treating boundaries as 1D signals, the model can flexibly "stretch" or "shrink" the bounding box in any direction independently. This allows the tracker to adapt to complex deformations that 2D filters often miss.

The IBCCF architecture: (a) standard 2D center region filtering and (b) the proposed multi-channel 1D boundary correlation filtering.

Near-Orthogonality Regularization

A key insight of this work is that the features representing the center of an object and its boundaries are naturally "nearly orthogonal"—meaning they provide non-overlapping, complementary information. The authors formalized this into a mathematical constraint (a regularization term) in their objective function. This forces the center and boundary filters to learn distinct, complementary features rather than redundant ones, ensuring that boundary filters focus on edges while the center filter focuses on the core appearance.

Optimization via ADMM

To solve the resulting complex optimization problem, the researchers employed the Alternating Direction Method of Multipliers (ADMM). This allows the model to iteratively update the center and boundary filters, typically converging to an optimal solution within five iterations.

Experimental Evaluation

The model was evaluated on several standard computer vision benchmarks, including OTB-2013, OTB-2015, Temple-Color, and the VOT series (2016/2017).

Key Results

  • Superior Accuracy: On OTB-2015, IBCCF achieved a 78.4% mean Overlap Precision (OP), significantly outperforming its baseline counterpart, HCF (65.6%), and other scale-adaptive trackers like DSST and SAMF.
  • Robustness to Rotation: In sequences involving in-plane and out-of-plane rotation—primary causes of aspect ratio changes—IBCCF showed gains of 2.5% and 6.4% respectively over standard deep-feature trackers.
  • Efficiency: While more complex than a basic CF tracker, the ADMM approach remains relatively efficient, running at approximately 1.25 frames per second (fps) on a high-end PC (Xeon CPU + GTX 1080).

Discussion and Future Work

The most significant contribution of IBCCF is the "divide and conquer" strategy of moving from 2D to 1D for boundary detection. Enforcing near-orthogonality ensures the filters do not interfere with one another, leading to a 4.7% gain in precision on challenging sequences.

However, limitations remain. At 1.25 fps, the current implementation is not yet fast enough for real-time applications like drone navigation without further optimization. Future work may involve integrating advanced CF techniques, such as spatial regularization or continuous convolution (as seen in C-COT), to further enhance performance and speed.

Summary updated

Abstract

The aspect ratio variation frequently appears in visual tracking and has a severe influence on performance. Although many correlation filter (CF)-based trackers have also been suggested for scale adaptive tracking, few studies have been given to handle the aspect ratio variation for CF trackers. In this paper, we make the first attempt to address this issue by introducing a family of 1D boundary CFs to localize the left, right, top, and bottom boundaries in videos. This allows us cope with the aspect ratio variation flexibly during tracking. Specifically, we present a novel tracking model to integrate 1D Boundary and 2D Center CFs (IBCCF) where boundary and center filters are enforced by a near-orthogonality regularization term. To optimize our IBCCF model, we develop an alternating direction method of multipliers. Experiments on several datasets show that IBCCF can effectively handle aspect ratio variation, and achieves state-of-the-art performance in terms of accuracy and robustness.
BibTeX
@inproceedings{li2017integrating,
  author = {Feng Li and Yingjie Yao and Peihua Li and David Zhang and Wangmeng Zuo and Ming-Hsuan Yang},
  title = {{Integrating Boundary and Center Correlation Filters for Visual Tracking with Aspect Ratio Variation}},
  booktitle = {Workshops @ ICCV},
  year = {2017},
  doi = {10.1109/iccvw.2017.234},
}
Figure