Abstract
Place recognition is a fundamental task for robot localization, typically requiring matching current sensor data to a pre-existing map. While Lidar-to-overhead imagery matching is highly beneficial for navigation in areas without prior Lidar maps, training such cross-modal systems usually depends on expensive, perfectly synchronized paired datasets. This paper presents a self-supervised framework that eliminates the need for time-synced training data. By integrating synthetic data generation, a novel "neural de-aliasing" technique for sequence alignment, and metric learning, the proposed method achieves localization performance approaching supervised methods using only unpaired sequences.
Introduction
Determining a robot's location by matching sensor data to a map is a core challenge in robotics. While Lidar-to-Lidar matching is well-established, matching Lidar scans to satellite imagery offers a significant advantage: it allows for navigation in environments where prior Lidar maps are unavailable. However, the primary bottleneck in developing these systems is the requirement for "paired data"—Lidar scans and satellite images centered at the exact same coordinates and perfectly synchronized via high-precision GPS/INS.
This research addresses the "unpaired" scenario, where a robot possesses a sequence of Lidar scans and a sequence of satellite images from the same general route but lacks explicit 1-to-1 correspondences between them.
Methodology
The proposed framework bridges the modality gap between Lidar and satellite imagery through a multi-step self-supervised pipeline.
1. Synthetic Lidar Generation
To bring both data sources into a similar visual domain, a CycleGAN is employed to transform satellite images into "synthetic" Lidar scans (polar 2D scans).
2. Initial Lidar Embedding and Neural De-aliasing
A descriptor space for real Lidar scans is initially learned using temporal consistency. When comparing real Lidar to synthetic Lidar, the resulting affinity matrix—representing the match quality between every frame—is often noisy. The authors introduce "neural de-aliasing," using a Pix2Pix network trained on simulated data to "clean" these matrices and enhance the signal-to-noise ratio.
3. Sequence Alignment and Metric Learning
Using the de-aliased matrix, a modified Smith-Waterman algorithm identifies "pseudo-pairs"—sequences of Lidar frames and satellite images that likely correspond to the same physical locations. These pairs serve as positive samples in a triplet loss framework to train the final cross-modal embedding space. To ensure rotation invariance, images are processed in polar coordinates, allowing the network to be rotation-invariant through simple vertical shifts.
Experimental Results
The method was evaluated on the Oxford Radar RobotCar and KITTI datasets, with satellite imagery queried from public sources. To simulate real-world constraints, the authors used asynchronous prior GPS measurements and varying sampling frequencies.
Key findings include:
- Performance: On the RobotCar dataset, the self-supervised method achieved a Top-1 retrieval rate (within 60m) of 45.5%, compared to 50.8% for the supervised baseline.
- Localization Robustness: When integrated into a Monte Carlo Localization (MCL) filter, the system successfully tracked vehicle positions over several kilometers with errors typically under 50 meters.
- Cross-Modal Versatility: The pipeline demonstrated success not only for Lidar but also for Radar-to-satellite matching.
Discussion and Conclusion
A central insight of this work is that while individual synthetic-to-real Lidar matches may be weak, the sequence of matches along a trajectory provides robust localization. The "learning to clean" approach for affinity matrices significantly outperforms traditional hand-crafted filtering.
However, challenges remain in repetitive environments (e.g., suburban areas with similar housing) where false-positive rates increase. Additionally, the domain gap between high-resolution real Lidar (like KITTI's 64-beam) and synthetic generation remains a hurdle. Future work will focus on narrowing this gap to improve the precision of initial affinity matrices.
Summary updated