Semi-distributed Cross-modal Air-Ground Relative Localization

Weining Lu, Deer Bin, Lian Ma, Ming Ma, Zhihao Ma, Xiangyang Chen, Longfei Wang, Yixiao Feng, Zhouxian Jiang, Yongliang Shi, Bin Liang

TL;DR

This paper introduces a novel semi-distributed framework for air-ground relative localization that decouples individual robot SLAM systems, enhancing flexibility and accuracy. By leveraging deep learning-based keypoints and descriptors, it achieves robust localization with significantly reduced communication bandwidth, making it ideal for resource-constrained collaborative tasks.

What Problem Does This Paper Address?

Accurate and flexible relative localization is crucial for collaborative tasks involving aerial and ground robots. Existing approaches often rely on tightly coupled, distributed multi-robot SLAM systems with similar sensor configurations. This tight coupling limits flexibility, making them unsuitable for heterogeneous systems and challenging in bandwidth-limited environments due to the high data transmission requirements (e.g., raw images or point clouds). The paper aims to overcome these limitations by proposing a semi-distributed framework that is decoupled from individual robot SLAM systems, allowing for greater flexibility and efficiency in air-ground collaborations.

Methods

The proposed framework is semi-distributed, meaning the UGV (Unmanned Ground Vehicle) and UAV (Unmanned Aerial Vehicle) perform their own SLAM, but their relative localization is handled in a decoupled manner.

  • Data Preprocessing: Both UGV and UAV extract deep learning-based keypoints and global descriptors from their sensor data. The UAV transmits its keypoints, descriptors, and global descriptor along with its visual-inertial odometry (VIO) pose. The UGV uses its camera, LiDAR, and IMU data, with its pose derived from LiDAR-inertial odometry (LIO) through linear interpolation.
  • Pixel-Point Correspondence: To obtain depth information for UGV keypoints, a sparse voxel representation (iVox) is used to efficiently query depth from the UGV's LiDAR-SLAM map.
  • Database Maintenance and Retrieval: A keyframe database is maintained on the UGV using an incremental Hierarchical Navigable Small Worlds (HNSW) structure. This database stores global descriptors from both UAV and UGV keyframes, enabling retrieval of co-visible historical frames for robust relative pose estimation.
  • Two-Stage Optimization: This is the core of the relative localization.
  • First Stage: A local Bundle Adjustment (BA) on the UGV's sliding window refines UGV camera poses and map point depths. This stage corrects errors arising from extrinsic parameter offsets, time unsynchronization, and the inherent inaccuracies of LIO interpolation.
  • Second Stage: This stage optimizes the relative pose between the UAV and UGV (TWAWG) and the UAV's camera poses simultaneously. It uses air-ground cross-platform dense feature correspondences and incorporates spatial geometric constraints and multi-view texture consistency. A relative pose constraint regularization term is also used to ensure compatibility across multiple air-ground keyframe pairs.

Datasets

The paper introduces two custom datasets designed for air-ground collaboration scenarios with varying levels of co-visibility. These datasets include synchronized camera, LiDAR, and IMU data, along with pre-calibrated parameters and ground truth trajectories obtained from a motion capture system.

Key Insights and Ideas

  • Decoupling for Flexibility: The most significant insight is that decoupling relative localization from individual robot SLAM systems dramatically increases flexibility. This allows heterogeneous robots with different sensors and SLAM algorithms to collaborate effectively.
  • Leveraging UGV's Payload: The UGV's superior payload capacity is utilized to integrate multiple sensors and computational power, enabling it to act as a central hub for cross-modal localization.
  • Efficient Communication: Transmitting only keypoint pixels and descriptors, rather than raw sensor data, drastically reduces communication bandwidth requirements, making the system practical for bandwidth-constrained environments.
  • Two-Stage Optimization for Accuracy: The two-stage optimization approach effectively addresses the inherent challenges of air-ground localization, such as lack of common field of view and sensor inaccuracies, leading to improved accuracy.
  • Yaw-Only Optimization: By aligning the map's z-axis with gravity, the rotational component of the relative pose estimation is simplified to a yaw angle optimization, improving efficiency.

Key Results

  • Accuracy: The proposed method achieves centimeter-level accuracy in relative pose estimation.
  • Efficiency: The system demonstrates high efficiency, with a favorable balance between precision and computational time.
  • Bandwidth Reduction: The keypoint-based communication strategy effectively constrains bandwidth to under 0.3 Mbps for acceptable precision, and around 2.18 Mbps for sub-centimeter accuracy with 1024 keypoints.
  • Performance Comparison: The method outperforms existing baselines like D2SLAM and TEASER in terms of stability, accuracy, and efficiency, especially in scenarios with limited common-view information.

Limitations and Future Work

  • Computational Cost: While efficient, the two-stage optimization process can lead to increased computational time as complexity rises. The authors note that performing relative localization once per second is sufficient for practical applications, achieving a balance.
  • Initialization: The paper mentions that D2SLAM struggles with initialization without sufficient common-view information. While their method is more robust, further investigation into initialization strategies for extremely challenging scenarios might be beneficial.
  • Future Work: The authors suggest that this work provides a promising direction for future air-ground collaboration, particularly in GPS-denied and bandwidth-limited environments, with potential applications in wilderness and post-disaster settings.

Summary updated

BibTeX
@inproceedings{lu2025semi,
  author = {Weining Lu and Deer Bin and Lian Ma and Ming Ma and Zhihao Ma and Xiangyang Chen and Longfei Wang and Yixiao Feng and Zhouxian Jiang and Yongliang Shi and Bin Liang},
  title = {{Semi-distributed Cross-modal Air-Ground Relative Localization}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2025},
  doi = {10.1109/iros60139.2025.11245976},
}
Figure