Abstract
Vision-based geo-localization for Unmanned Aerial Vehicles (UAVs) provides a critical alternative to GNSS in GPS-denied environments. While deep learning has framed this as an image retrieval task, existing datasets rely on unrealistic "perfectly aligned" drone-satellite image pairs. This work introduces GTA-UAV, a large-scale dataset generated from game data that features contiguous areas, varying altitudes, and camera attitudes. We propose a more practical Partial N-to-N Matching task and a Weighted-InfoNCE contrastive learning approach to handle overlapping views. Experiments show that our method significantly improves localization accuracy and generalizes well to real-world scenarios.
The Alignment Gap in UAV Geo-Localization
Current UAV geo-localization research often operates under a "strong assumption": that every drone-view query has a perfectly centered, one-to-one corresponding satellite image in the reference database. In real-world flight, drones capture ground views at arbitrary altitudes and angles, rarely aligning perfectly with pre-defined satellite tiles. This discrepancy creates a significant performance gap when models move from laboratory benchmarks to field deployment.
To address this, we shift the paradigm toward Partial N-to-N Matching. In this scenario, a drone image may only partially overlap with multiple satellite tiles, requiring the model to understand spatial relationships beyond simple identity matching. This approach expands image-level retrieval to actual localization measured in meters.
Methodology: Embracing Partial Overlaps
We introduce two primary innovations to train models for this realistic environment:
- Weighted-InfoNCE Loss: Traditional contrastive learning uses binary labels (match or no-match). We instead use the Intersection over Union (IoU) between the drone's field of view and the satellite tile to weight the loss. This allows the model to learn that a high-overlap pair provides a stronger positive signal than a marginal overlap.
- Mutually Exclusive Sampling (MES): To prevent "false negatives" during batch training—where two drone images covering the same area might be treated as negatives for each other—we use a graph-based sampling strategy to ensure every sample in a batch is spatially independent.
The system utilizes a Vision Transformer (ViT-Base) backbone. During inference, drone query features are matched against a pre-computed feature gallery of geo-tagged satellite images to retrieve the Top-K most likely locations.
The GTA-UAV Dataset
High-fidelity simulation via Grand Theft Auto V (GTA V) allows for the creation of a massive, privacy-compliant dataset with perfect ground truth.
- Scale and Diversity: GTA-UAV includes 33,763 drone images and 14,640 satellite tiles covering 81.3 $km^2$ of diverse terrain, including urban areas, mountains, and coasts.
- Realism: The dataset captures flight altitudes from 80m to 650m and includes varying camera pitch, roll, and yaw, forcing models to handle geometric distortions and varying scales.
Key Results and Real-World Transfer
Our approach demonstrates substantial improvements in both retrieval and actual distance-based localization:
- Accuracy Boost: On unseen "Cross-Area" tasks, our method improved Recall@1 by 20.08% compared to standard contrastive learning.
- Precision: The average localization error (Dis@1) was reduced by 234.36 meters in cross-area settings.
- Generalization: When tested on the real-world UAV-VisLoc dataset without further fine-tuning, the model pre-trained on GTA-UAV significantly outperformed those trained on traditional datasets like University-1652, proving the effectiveness of game-based synthetic data for real-world geographic feature extraction.
Conclusion
By moving away from idealized assumptions and leveraging high-fidelity game data, Game4Loc provides a more robust foundation for UAV navigation. Future work will focus on narrowing the "sim-to-real" visual gap and achieving fine-grained, sub-meter localization.
Summary updated