Siamese Object Tracking for Vision-Based UAM Approaching with Pairwise Scale-Channel Attention

Guangze Zheng, Changhong Fu, Junjie Ye, Bowen Li, Geng Lu, Jia Pan

Abstract

Autonomous Unmanned Aerial Manipulators (UAMs) require high-precision tracking during the "approaching" phase to facilitate successful grasping and interaction. However, rapid changes in target scale and the need for real-time processing on embedded hardware present significant challenges. This paper introduces SiamSA, a Siamese tracking framework featuring a novel Pairwise Scale-Channel Attention (PSA) mechanism. By integrating Scale-Equivariant Convolutions (SEC) and a Scale-Aware Anchor Proposal Network (SA-APN), SiamSA effectively manages extreme scale variations. Evaluated on the new UAMT100 benchmark, the proposed tracker achieves a 10.6% improvement in success rate over state-of-the-art baselines while maintaining a high-speed performance of 122 FPS.

Introduction

Unmanned Aerial Manipulators (UAMs)—drones equipped with robotic arms—are increasingly utilized for complex tasks such as autonomous pick-and-place operations and infrastructure maintenance. A critical component of these operations is the visual approaching phase, where the UAM must accurately track a target object to position itself for manipulation.

Standard object trackers often fail in this context due to two primary factors. First, Extreme Scale Variation (SV): as the UAM closes the distance to a target, the object's size in the camera frame increases exponentially, often exceeding the capacity of general-purpose trackers. Second, Model Rigidity: many existing UAM tracking solutions rely on rigid, model-based methods that require predefined labels, limiting their ability to track arbitrary objects in dynamic environments. SiamSA addresses these issues by treating scale as a primary attention dimension within a flexible Siamese framework.

Methodology

The SiamSA architecture is built upon three core technical innovations designed to enhance scale robustness without sacrificing computational efficiency.

1. Scale-Equivariant Convolution (SEC)

Traditional convolutions are translation-invariant but struggle with scale changes. SiamSA replaces standard initial layers with Scale-Equivariant Convolutions. This provides the network with a "preliminary perception" of scale by adding a dedicated scale dimension to the image features, allowing the model to recognize patterns across different sizes more effectively from the outset.

2. Pairwise Scale-Channel Attention Network (PSAN)

While typical attention mechanisms focus on spatial ("where") or channel ("what") information, the PSAN module introduces scale as a primary dimension. By utilizing both self-attention and cross-attention, the network identifies which feature channels are most sensitive to scale changes. The "pairwise" structure refers to the simultaneous processing of correlation results (similarity between target and search area) and the feature maps themselves, ensuring that scale information is integrated throughout the tracking pipeline.

3. Scale-Aware Anchor Proposal Network (SA-APN)

In Siamese tracking, anchors are used to propose potential object locations. The SA-APN improves this process by aggregating features from both shallow layers (which preserve high-resolution spatial details) and deep layers (which provide rich semantic information). This multi-level fusion ensures that the proposed anchors are inherently scale-aware, leading to more accurate bounding box predictions during rapid approach maneuvers.

The UAMT100 Benchmark

To rigorously test the framework, the researchers introduced UAMT100, a benchmark specifically tailored for UAM approaching scenarios.

  • Data Volume: 100 image sequences comprising 35,000 frames.
  • Source: Recorded directly from a flying UAM platform.
  • Unique Challenges: Includes specific attributes like "UAM attack" (where external interference, such as a stick, disrupts the drone) and extreme scale variation typical of robotic grasping tasks.

Experimental Results

Performance Metrics

SiamSA was evaluated against 11 state-of-the-art trackers. On the UAMT100 benchmark, it achieved:

  • Success Rate (AUC): A 10.6% gain over the baseline.
  • Normalized Precision (NP): A 9.2% improvement.
  • Scale Robustness: In scenarios specifically isolated for scale variation, SiamSA outperformed the second-best tracker by 10.6%.

Efficiency and Deployment

Despite the sophisticated attention mechanisms, SiamSA is highly optimized. Using a lightweight AlexNet backbone, it achieves 122 FPS on a desktop GPU. More importantly, it maintains a real-time rate of over 10 FPS when deployed on an onboard NVIDIA Jetson Xavier NX, proving its viability for edge computing on physical robotic platforms.

Conclusion

SiamSA represents a significant step forward in vision-based UAM navigation. By prioritizing scale as a fundamental dimension of attention and introducing the UAMT100 benchmark, this research provides a robust foundation for autonomous aerial manipulation. Future work will focus on integrating this perception framework directly into the UAM's flight control loop for end-to-end autonomous grasping and exploring performance in diverse outdoor environments with varying lighting and weather conditions.

Summary updated

Abstract

Although the manipulating of the unmanned aerial manipulator (UAM) has been widely studied, vision-based UAM approaching, which is crucial to the subsequent manipulating, generally lacks effective design. The key to the visual UAM approaching lies in object tracking, while current UAM tracking typically relies on costly model-based methods. Besides, UAM approaching often confronts more severe object scale variation issues, which makes it inappro-priate to directly employ state-of-the-art model-free Siamese-based methods from the object tracking field. To address the above problems, this work proposes a novel Siamese network with pairwise scale-channel attention (SiamSA) for vision-based UAM approaching. Specifically, SiamSA consists of a pairwise scale-channel attention network (PSAN) and a scale-aware anchor proposal network (SA-APN). PSAN acquires valuable scale information for feature processing, while SA-APN mainly attaches scale awareness to anchor proposing. Moreover, a new tracking benchmark for UAM approaching, namely UAMT100, is recorded with 35K frames on a flying UAM platform for evaluation. Exhaustive experiments on the benchmarks and real-world tests validate the efficiency and practicality of SiamSA with a promising speed. Both the code and UAMT100 benchmark are now available at https://github.com/vision4robotics/SiamSA.
BibTeX
@inproceedings{zheng2022siamese,
  author = {Guangze Zheng and Changhong Fu and Junjie Ye and Bowen Li and Geng Lu and Jia Pan},
  title = {{Siamese Object Tracking for Vision-Based UAM Approaching with Pairwise Scale-Channel Attention}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2022},
  doi = {10.1109/iros47612.2022.9982189},
}
Figure