Abstract
Robust and accurate visual tracking is one of the most challenging computer vision problems. Due to the inherent lack of training data, a robust approach for constructing a target appearance model is crucial. Recently, discriminatively learned correlation filters (DCF) have been successfully applied to address this problem for tracking. These methods utilize a periodic assumption of the training samples to efficiently learn a classifier on all patches in the target neighborhood. However, the periodic assumption also introduces unwanted boundary effects, which severely degrade the quality of the tracking model. We propose Spatially Regularized Discriminative Correlation Filters (SRDCF) for tracking. A spatial regularization component is introduced in the learning to penalize correlation filter coefficients depending on their spatial location. Our SRDCF formulation allows the correlation filters to be learned on a significantly larger set of negative training samples, without corrupting the positive samples. We further propose an optimization strategy, based on the iterative Gauss-Seidel method, for efficient online learning of our SRDCF. Experiments are performed on four benchmark datasets: OTB-2013, ALOV++, OTB-2015, and VOT2014. Our approach achieves state-of-the-art results on all four datasets. On OTB-2013 and OTB-2015, we obtain an absolute gain of 8.0% and 8.2% respectively, in mean overlap precision, compared to the best existing trackers.
The Boundary Effect Challenge
Visual tracking—estimating the trajectory of a target in a video given only its initial location—is notoriously difficult due to limited training data and the need for real-time processing. Discriminative Correlation Filters (DCF) became popular because they are incredibly fast, leveraging the Fast Fourier Transform (FFT) to learn a classifier in the frequency domain.
However, the FFT assumes the signal is periodic. In tracking, this means the algorithm "sees" the image patch as a repeating tile. This leads to two major issues:
- Boundary Effects: The model is trained on "synthetic" negative samples created by the periodic shifts that don't exist in the real image, leading to an inaccurate representation of the background.
- Restricted Search Area: Because of these boundary artifacts, standard trackers can only look at a very small region around the target. If the target moves quickly, the tracker easily loses it.
Methods: Spatially Regularized DCF (SRDCF)
The authors propose SRDCF, which modifies the standard DCF objective function by adding a spatial regularization component to mitigate these boundary effects.
- Spatial Regularization: They introduce a weight function $w$ that varies across the filter. This function has low values in the center (the target) and very high values toward the edges (the background). By multiplying the filter coefficients by this weight during training, the optimization is forced to concentrate the filter's energy on the target, effectively "muting" the background.
- Larger Training Patches: Because the regularization handles the background clutter, the authors can use training patches that are significantly larger than standard DCF methods. This provides more real negative samples for better discrimination.
- Iterative Optimization: Adding this spatial weight breaks the simple "closed-form" solution usually found in the Fourier domain. To keep the tracker efficient, the authors employ the Gauss-Seidel method. This iterative approach solves the resulting sparse linear system much faster than standard solvers.
- Sub-grid Detection: To improve accuracy beyond the pixel level, they use a trigonometric interpolation strategy to find the maximum detection score with sub-pixel precision.
Key Results
SRDCF set a new standard for performance at the time of publication, showing significant improvements in scenarios involving fast motion and occlusions.
- OTB Benchmarks: The method achieved a Mean Overlap Precision of 78.1% on OTB-2013 and 72.9% on OTB-2015. This represented an absolute gain of 8.0% and 8.2% respectively over the previous best trackers like MEEM and SAMF.
- Efficiency: Despite the iterative optimization, the Matlab implementation ran at 5 frames per second (FPS). While not as fast as pure DCF methods (which can exceed 100 FPS), it was efficient enough for many practical applications given the massive jump in accuracy.
Conclusion and Future Work
SRDCF successfully addresses the fundamental "periodic trap" of correlation filters. By penalizing coefficients near the boundaries, the tracker behaves more like a traditional sliding-window classifier while retaining the computational advantages of the Fourier domain.
Future work includes reducing the computational overhead to reach real-time speeds (30+ FPS) and integrating deep convolutional features, which the authors suggest could further push the boundaries of tracking performance.
Summary updated