Introduction
Visual object tracking is a fundamental computer vision task that requires identifying and locating a target object across a video sequence. Modern high-performance trackers typically employ a Siamese architecture, which consists of two parallel branches: a template branch that extracts features from the target object in the initial frame, and a search branch that processes a larger region in subsequent frames. The core of these trackers is the fusion mechanism—traditionally cross-correlation (XCorr)—which computes a similarity map to localize the target.
The Limitations of Cross-Correlation
Despite its popularity, standard cross-correlation and its depth-wise variant (DW-XCorr) suffer from several inherent weaknesses. First, XCorr is a parameter-free, handcrafted operation. Because it lacks learnable weights, it cannot adapt to the complex semantic nuances present in large-scale tracking datasets. Second, XCorr is frequently "fooled" by distractors—background objects that share visual similarities with the target. Finally, XCorr tends to activate only a sparse subset of feature channels, resulting in blurry response maps that hinder precise boundary localization.
The Asymmetric Convolution Module (ACM)
To overcome these hurdles, researchers have proposed the Asymmetric Convolution Module (ACM) as a learnable alternative to the static cross-correlation layer. The primary challenge in replacing XCorr is the size asymmetry between the template (e.g., $7 \times 7$) and the search region (e.g., $31 \times 31$).
The ACM addresses this by mathematically decomposing the fusion process. Instead of a direct convolution, the module treats fusion as a concatenation followed by a convolution. This is implemented via two independent convolutions—one for the template and one for the search region—which are then combined using broadcasting summation. This structure allows the network to learn specific weights for how features from both branches should interact. Furthermore, the flexibility of ACM allows for the integration of non-visual priors, such as the initial bounding-box dimensions, providing the tracker with essential scale information.
Experimental Evaluation
The ACM was integrated into three prominent baseline trackers: SiamFC, SiamRPN++, and SiamBAN. These models were evaluated across six major benchmarks, including LaSOT, TrackingNet, VOT2016/2019, OTB-100, and UAV123.
Key Insights and Results
The transition from handcrafted math to learnable fusion yielded significant performance gains. On the LaSOT benchmark, the ACM-enhanced tracker achieved a 5.8% improvement in Success (AUC) over the baseline. On TrackingNet, the SiamBAN-ACM variant reached a state-of-the-art success score of 0.753, outperforming established models like KYS and DiMP.
Qualitative analysis reveals that ACM utilizes a much wider diversity of feature channels compared to DW-XCorr. While DW-XCorr activates only a few channels, ACM leverages many, with different channels learning to identify distinct parts of the target. This results in sharper response maps with clear object boundaries, which is critical for the regression head to accurately predict bounding boxes.
Despite the addition of learnable parameters, the module remains highly efficient. Trackers equipped with ACM maintain real-time speeds of 40–41 FPS on a standard GTX 1080Ti GPU, ensuring that the accuracy gains do not come at the cost of practical utility.
Conclusion
The Asymmetric Convolution Module represents a significant step forward in Siamese tracking by replacing static operations with a learnable fusion mechanism. While the current implementation relies on the accuracy of the initial bounding box for its non-visual priors, the robust performance across multiple benchmarks suggests that learnable asymmetric fusion is a powerful tool for modern visual tracking. Future work may explore the application of ACM to related tasks such as object detection and re-identification.
Summary updated