AGEN: Adaptive Error Control-Driven Cross-View Geo-Localization Under Extreme Weather Conditions

Mengmeng Xu, Hongxiang Lv, Hai Zhu, Enlai Dong, Fei WuSensors 2025 · DOI

TL;DR

This paper introduces AGEN, a novel framework for robust cross-view geo-localization under extreme weather conditions. By combining the powerful global feature extraction of DINOv2 with the fine-grained detail capture of LPN, and introducing an Adaptive Error Control (AEC) mechanism to dynamically optimize the loss function, AGEN significantly improves accuracy and resilience in challenging environments.

What Problem Does This Paper Address?

Geo-localization, the task of determining an object's geographic location from an image, is crucial for applications like drone navigation and autonomous vehicles. Cross-view geo-localization, which matches images from different perspectives (e.g., drone views to satellite maps), offers a GPS-free alternative. However, existing methods struggle significantly when faced with extreme weather conditions like heavy rain, snow, or fog. These conditions degrade image quality, leading to a sharp drop in localization accuracy. While prior work focused on image denoising, this often results in the loss of critical image features. This paper aims to bridge this gap by developing a system that is inherently robust to these adverse weather conditions, maintaining high accuracy without solely relying on pre-processing.

Methods

The core of AGEN is a twin network architecture that leverages a DINOv2 backbone for its strong global feature extraction capabilities. To address DINOv2's limitations in capturing fine-grained details, the authors integrate a Local Pattern Network (LPN) module. LPN helps extract more localized and detailed features, crucial for accurately interpreting complex scenes.

The most innovative aspect is the Adaptive Error Control (AEC) module. This module employs fuzzy PID control to dynamically adjust the loss function's weights during training. Instead of using static loss weights, AEC analyzes historical error information and its temporal evolution to adaptively emphasize more challenging samples or training stages. This dynamic adjustment aims to improve training stability and model robustness, especially under the noisy conditions introduced by extreme weather. The authors also employed a data augmentation strategy, creating an expanded dataset with nine different weather conditions.

Datasets

The paper utilizes and expands upon several key datasets:

  • University-1652: A foundational dataset with 1652 university locations, used for general cross-view geo-localization research.
  • University160k-WX: Specifically designed to address extreme weather conditions, this dataset introduces weather noise (fog, rain, snow, etc.) to drone-view images. The authors further augment this dataset to create University1652-plus, which includes nine distinct weather conditions for drone-view images.
  • SUES-200: A dataset with 24,120 images captured by UAVs at various altitudes, offering a diverse range of scenes and realistic conditions.

Key Insights and Ideas

  • Synergy of Global and Local Features: The paper highlights that combining powerful global feature extractors (like DINOv2) with modules designed for fine-grained local details (like LPN) is crucial for robust scene understanding, especially in complex environments.
  • Dynamic Loss Optimization is Key for Robustness: The core insight is that static loss functions are insufficient for extreme weather. An adaptive mechanism like AEC, which learns from past errors and trends, can significantly improve a model's ability to handle noisy data and challenging training dynamics.
  • Data Augmentation Beyond Simple Denoising: Simply denoising images can remove valuable information. The paper's approach of augmenting data with various weather conditions and then training a robust model directly on this augmented data proves more effective than relying solely on pre-processing.

Key Results

  • State-of-the-Art Performance: AGEN achieves state-of-the-art Recall@1 accuracy on multiple benchmarks.
  • On the University-1652 dataset, AGEN achieves 95.43% Recall@1 for drone-view target localization and 96.72% for drone navigation.
  • Under extreme weather conditions on the University160k-WX dataset, AGEN achieves 91.71% Recall@1.
  • On the SUES-200 dataset, AGEN demonstrates strong performance across different altitudes, reaching up to 97.12% Recall@1.
  • AEC Module Effectiveness: Ablation studies show that the AEC module consistently improves performance across different backbone architectures (VGG16, ResNet34/50, ViT), demonstrating its general applicability.
  • Robustness to Weather: The proposed method shows superior generalization and robustness compared to other methods when tested on challenging weather conditions.

Limitations and Future Work

  • Limited Fine-Grained Analysis: The test set for University160k-WX uses a masked evaluation protocol without specific weather labels, limiting fine-grained analysis of performance under individual weather types (e.g., fog vs. rain). Future work could involve datasets with explicit weather annotations.
  • Synthetic vs. Real-World Data Gap: The reliance on synthetic augmentation for weather conditions might introduce a domain gap compared to real-world data. Future research could focus on collecting and labeling real adverse weather imagery or exploring generative models (like diffusion models) to synthesize more realistic weather data and improve clean image restoration.
  • Limited Weather Types: While the current weather types cover typical scenarios, performance might degrade in more complex or extreme conditions. Further exploration of generative models is suggested to handle a wider range of weather complexities.

Summary updated

Abstract

Cross-view geo-localization is a task of matching the same geographic image from different views, e.g., drone and satellite. Due to its GPS-free advantage, cross-view geo-localization is gaining increasing research interest, especially in drone-based localization and navigation applications. In order to guarantee system accuracy, existing methods mainly focused on image augmentation and denoising while still facing performance degradation when extreme weather conditions are considered. In this paper, we propose a robust end-to-end image retrieval framework, AGEN, serving for cross-view geo-localization under extreme weather conditions. Inspired by the strengths of the DINOv2 network, particularly its strong performance in global feature extraction, while acknowledging its limitations in capturing fine-grained details, we integrate the DINOv2 network with the Local Pattern Network (LPN) algorithm module to extract valuable classification features more efficiently. Additionally, to further enhance model robustness, we innovatively introduce an Adaptive Error Control (AEC) module based on fuzzy control to optimize the loss function dynamically. Specifically, by adjusting loss weights adaptively, the AEC module allows the model to better handle complex and challenging scenarios. Experimental results demonstrate that AGEN achieves a Recall@1 accuracy of 91.71% on the University160k-WX dataset under extreme weather conditions. Through extensive experiments on two well-known public datasets, i.e., University-1652 and SUES-200, AGEN achieves state-of-the-art Recall@1 accuracy in both drone-view target localization tasks and drone navigation tasks, outperforming existing models. In particular, on the University-1652 dataset, AGEN reaches 95.43% Recall@1 in the drone-view target localization task, showcasing its superior capability in handling challenging scenarios.
BibTeX
@inproceedings{xu2025agen,
  author = {Mengmeng Xu and Hongxiang Lv and Hai Zhu and Enlai Dong and Fei Wu},
  title = {{AGEN: Adaptive Error Control-Driven Cross-View Geo-Localization Under Extreme Weather Conditions}},
  booktitle = {Sensors},
  year = {2025},
  doi = {10.3390/s25123749},
}
Figure