Towards Label-Efficient Geospatial Foundation Model Adaptation via Positive-Unlabeled Learning for Building Damage Mapping

Hengwei Zhao, Qirui Wang, Zhuo Zheng, Xiaoyan Lu, Rusty Feagin, Wenzhe JiaoarXiv 2026 · arXiv

TL;DR

PUMA (Positive-Unlabeled GFM Adaptation) is a new framework that allows Geospatial Foundation Models (GFMs) to accurately map building damage using only a handful of confirmed "damaged" examples and a large pool of unlabeled data. By combining non-contrastive representation learning with a novel label disambiguation strategy, PUMA achieves high performance with 40x fewer labels than standard supervised methods, making it ideal for rapid disaster response.

What Problem Does This Paper Address?

When a disaster strikes, emergency responders need accurate building damage maps immediately. While Geospatial Foundation Models (GFMs) provide a powerful starting point, they must be fine-tuned for specific events. However, standard fine-tuning is a "labeling bottleneck": it requires large amounts of high-quality annotations for both damaged and undamaged buildings—data that is rarely available in the chaotic first hours of a disaster.

Most real-world scenarios provide only Positive-Unlabeled (PU) data: a few confirmed damaged buildings (positives) and a vast number of buildings with unknown status (unlabeled). Existing adaptation techniques often struggle in this setting, either requiring a pre-estimated "class prior" (the exact percentage of damaged buildings, which is usually unknown) or failing to learn distinct enough features to separate damaged from undamaged structures.

Methods

The authors propose PUMA, which integrates two mutually reinforcing modules to bridge the gap between weak supervision and foundation model adaptation:

  1. NeWER (Non-contrastive Weakly supervised Representation refinement): This module focuses on learning better features. It uses a non-contrastive architecture (similar to SimSiam) to avoid "intra-class noise" (erroneously pushing similar buildings apart). To handle "inter-class noise" (pulling different classes together due to wrong labels), the authors introduced two new loss functions ($L_{rep}^{sqrt}$ and $L_{rep}^{log}$) designed to suppress the gradients of noisy pairs, ensuring the model focuses on reliable data.
  2. LaTTE (Latent-state-based label disambiguation): This module progressively refines the labels of the unlabeled pool. It treats the model's confidence as a "latent state" to identify reliable undamaged buildings. Crucially, it is "reversible"—if the model becomes less confident about a previously assigned label, it can revert to a soft estimate rather than sticking to a hard, potentially wrong, decision.
The PUMA framework overview showing the interaction between representation refinement and label disambiguation.

The entire process is theoretically grounded in a Generalized Expectation-Maximization (GEM) framework, where representation learning and label refinement iteratively improve each other.

Datasets

The researchers evaluated PUMA across diverse disaster types and geographies:

  • xBD Dataset: A global benchmark for building damage, specifically subsets for Hurricane Florence, the Joplin Tornado, and the Santa Rosa Wildfires.
  • ABCD Dataset: The AIST Building Change Detection dataset, focusing on the 2011 Tohoku tsunami in Japan.
  • Real-world Case Study: Data from the 2025 Los Angeles wildfires (Eaton and Palisades fires), used to test the model's performance under extreme label constraints (as few as 50 damaged building samples).

Key Insights and Ideas

  • The Gradient Perspective: The authors identified that standard supervised losses are easily biased by noisy labels in a PU setting. Their proposed square-root and log-based losses mathematically ensure that "clean" pairs contribute more to the learning process than "noisy" ones.
  • Class Prior Independence: Unlike many PU learning methods that fail if you don't know the percentage of damaged buildings ($\pi_p$), PUMA's latent-state approach bypasses this requirement. This makes it significantly more practical for "unseen" disasters where such statistics are unavailable.
  • Discriminative Power: As shown in the t-SNE visualizations (Figure 1), PUMA creates much tighter clusters and a clearer separation between classes compared to standard cross-entropy or other PU methods.
t-SNE visualization showing how PUMA (d) achieves superior class separation compared to baselines.

Key Results

  • Superior Accuracy: PUMA consistently outperformed state-of-the-art PU learning and GFM fine-tuning methods across all benchmarks. For instance, on the tsunami dataset, it improved the F1-score by nearly 6 percentage points over the next best method.
  • Extreme Label Efficiency: In the Los Angeles wildfire application, PUMA achieved an F1-score of 92.81% using only 50 damaged samples. To reach similar performance, standard supervised fine-tuning required 40 times more labels.
  • Robustness: The framework proved compatible with various Parameter-Efficient Fine-Tuning (PEFT) strategies like LoRA and VPT, as well as different backbones (CLIP, RemoteCLIP, etc.), proving it isn't tied to one specific architecture.

Limitations and Future Work

While PUMA excels at binary classification (damaged vs. undamaged), the authors note that real-world damage assessment often requires more granular levels (e.g., minor damage, major damage, destroyed). Future work aims to extend the framework to multi-class damage assessment and explore its application in other weakly supervised remote sensing tasks, such as flood mapping or forest disturbance detection.

Summary updated

Abstract

Geospatial foundation models (GFMs) are reshaping remote sensing by shifting the field from task-specific modeling toward general-purpose geospatial intelligence. Mapping the damage to buildings stands to benefit substantially from this shift, as GFMs pretrained on large-scale Earth observation data provide generalizable geospatial representations that can facilitate effective responses to sudden-onset disasters. Although parameter-efficient fine-tuning (PEFT) facilitates GFM adaptation, parameter efficiency does not necessarily translate into label efficiency. Building damage mapping remains challenging due to scarce pre-existing labeled data or limited time for annotation. When weak supervision is introduced to alleviate the labeling bottleneck, we identify the difficulty of learning sufficiently discriminative representations as a key challenge for label-efficient GFM adaptation. To address this challenge, we propose a Positive-Unlabeled GFM Adaptation (PUMA) framework that requires annotations for only a small number of damaged buildings. More specifically, PUMA integrates non-contrastive weakly supervised representation refinement to mitigate the impact of noisy supervision on representation learning from a gradient perspective, and latent-state-based label disambiguation to enable reversible supervision refinement without requiring class prior estimation. Extensive experiments on disaster benchmark datasets from around the world demonstrated the effectiveness and robustness of PUMA. A real-world application to the 2025 Los Angeles wildfires further highlighted its potential to accurately map damage under highly limited supervision. More broadly, these findings suggest that weakly supervised PEFT offers a label-efficient pathway from GFMs to real-world applications.
BibTeX
@misc{zhao2026towards,
  author = {Hengwei Zhao and Qirui Wang and Zhuo Zheng and Xiaoyan Lu and Rusty Feagin and Wenzhe Jiao},
  title = {{Towards Label-Efficient Geospatial Foundation Model Adaptation via Positive-Unlabeled Learning for Building Damage Mapping}},
  year = {2026},
  eprint = {2611.08561},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.08561},
}
Figure