CurriculumLoc: Enhancing Cross-Domain Geolocalization through Multi-Stage Refinement

Boni Hu, Lin Chen, Runjian Chen, Shuhui Bu, Pengcheng Han, Haowei Li

Abstract

Visual geolocalization for Unmanned Aerial Vehicles (UAVs) remains a significant challenge due to the drastic domain gap between low-altitude drone imagery and high-altitude satellite maps. We present CurriculumLoc, a multi-stage refinement pipeline inspired by curriculum learning. By transitioning from coarse global retrieval to fine-grained geometric pose estimation using Swin Transformer-based descriptors and adaptive filtering, CurriculumLoc achieves state-of-the-art performance, reaching a 94.5% Recall@1 on benchmark datasets and sub-10-meter localization accuracy.

Introduction

Visual geolocalization involves determining a camera's precise coordinates by matching a query image against a geo-tagged reference database. For UAVs, this task is complicated by viewpoint variations, seasonal changes, and lighting differences between drone-captured photos and satellite imagery. Existing methods typically struggle to balance the speed of retrieval-based approaches with the precision of structure-based matching. CurriculumLoc addresses this by mimicking human learning patterns—starting with broad scene recognition and narrowing down to precise geometric details.

Methodology: The CurriculumLoc Pipeline

The proposed system operates through a five-stage "curriculum" designed to progressively refine the localization estimate.

The detailed pipeline of CurriculumLoc, including Swin-Descriptors and PnP-based pose estimation.
  1. Initial Retrieval: A global descriptor (NetVLAD) identifies the top 50 satellite candidates to constrain the search space.
  2. Swin-Descriptors: A symmetric Encoder-Decoder architecture utilizing Swin Transformers extracts local features with global context. This overcomes the limited receptive fields of traditional CNNs.
  3. Soft Detection Score Map: The model learns to prioritize distinctive landmarks while ignoring repetitive or transient features like roads or vegetation.
  4. Adaptive Reranking: A dynamic thresholding mechanism, the Adaptive Distance Filter, statistically evaluates image pairs to improve matching resilience across different sensors.
  5. Pose Estimation: The final stage solves the Perspective-n-Point (PnP) problem by establishing 2D-3D correspondences, yielding exact latitude, longitude, and altitude.

Key Insights and Visual Verification

A critical component of CurriculumLoc is its ability to generate robust keypoints despite significant domain shifts. By leveraging Transformers, the system ensures that local points are contextually aware of the broader scene.

Visualizing cross-domain matching and soft detection scores between UAV and satellite images.

This global awareness prevents mismatches in areas with repetitive textures. The "soft" score map effectively filters out noise, focusing the geometric verification on stable, high-confidence landmarks.

Experimental Results

The system was evaluated on the ALTO benchmark and a new, challenging TerraTrack dataset.

  • ALTO Dataset: CurriculumLoc achieved a 94.5% Recall@1 at a 50m threshold, significantly outperforming previous state-of-the-art methods like DELG (35.9% at 20m).
  • TerraTrack Dataset: The model demonstrated high robustness to viewpoint shifts and repetitive textures, achieving 62.7% Recall@1 at 20m.
  • Precision: Mean localization error was reduced to under 10 meters, proving the system's viability for GPS-denied navigation.

Conclusion and Future Work

CurriculumLoc demonstrates that a multi-stage, curriculum-based approach can effectively bridge the cross-domain gap in UAV geolocalization. While the current pipeline is computationally intensive, future work will focus on optimizing the backbone for real-time performance and integrating the system into full SLAM frameworks for continuous autonomous flight.

Summary updated

BibTeX
@article{hu2024curriculumloc,
  author = {Boni Hu and Lin Chen and Runjian Chen and Shuhui Bu and Pengcheng Han and Haowei Li},
  title = {{CurriculumLoc: Enhancing Cross-Domain Geolocalization through Multi-Stage Refinement}},
  journal = {IEEE Trans. Geosci. Remote. Sens.},
  year = {2024},
  doi = {10.1109/tgrs.2024.3380191},
}
Figure