TL;DR
PUMA (Positive-Unlabeled GFM Adaptation) is a new framework that allows Geospatial Foundation Models (GFMs) to accurately map building damage using only a handful of confirmed "damaged" examples and a large pool of unlabeled data. By combining non-contrastive representation learning with a novel label disambiguation strategy, PUMA achieves high performance with 40x fewer labels than standard supervised methods, making it ideal for rapid disaster response.
What Problem Does This Paper Address?
When a disaster strikes, emergency responders need accurate building damage maps immediately. While Geospatial Foundation Models (GFMs) provide a powerful starting point, they must be fine-tuned for specific events. However, standard fine-tuning is a "labeling bottleneck": it requires large amounts of high-quality annotations for both damaged and undamaged buildings—data that is rarely available in the chaotic first hours of a disaster.
Most real-world scenarios provide only Positive-Unlabeled (PU) data: a few confirmed damaged buildings (positives) and a vast number of buildings with unknown status (unlabeled). Existing adaptation techniques often struggle in this setting, either requiring a pre-estimated "class prior" (the exact percentage of damaged buildings, which is usually unknown) or failing to learn distinct enough features to separate damaged from undamaged structures.
Methods
The authors propose PUMA, which integrates two mutually reinforcing modules to bridge the gap between weak supervision and foundation model adaptation:
- NeWER (Non-contrastive Weakly supervised Representation refinement): This module focuses on learning better features. It uses a non-contrastive architecture (similar to SimSiam) to avoid "intra-class noise" (erroneously pushing similar buildings apart). To handle "inter-class noise" (pulling different classes together due to wrong labels), the authors introduced two new loss functions ($L_{rep}^{sqrt}$ and $L_{rep}^{log}$) designed to suppress the gradients of noisy pairs, ensuring the model focuses on reliable data.
- LaTTE (Latent-state-based label disambiguation): This module progressively refines the labels of the unlabeled pool. It treats the model's confidence as a "latent state" to identify reliable undamaged buildings. Crucially, it is "reversible"—if the model becomes less confident about a previously assigned label, it can revert to a soft estimate rather than sticking to a hard, potentially wrong, decision.
The entire process is theoretically grounded in a Generalized Expectation-Maximization (GEM) framework, where representation learning and label refinement iteratively improve each other.
Datasets
The researchers evaluated PUMA across diverse disaster types and geographies:
- xBD Dataset: A global benchmark for building damage, specifically subsets for Hurricane Florence, the Joplin Tornado, and the Santa Rosa Wildfires.
- ABCD Dataset: The AIST Building Change Detection dataset, focusing on the 2011 Tohoku tsunami in Japan.
- Real-world Case Study: Data from the 2025 Los Angeles wildfires (Eaton and Palisades fires), used to test the model's performance under extreme label constraints (as few as 50 damaged building samples).
Key Insights and Ideas
- The Gradient Perspective: The authors identified that standard supervised losses are easily biased by noisy labels in a PU setting. Their proposed square-root and log-based losses mathematically ensure that "clean" pairs contribute more to the learning process than "noisy" ones.
- Class Prior Independence: Unlike many PU learning methods that fail if you don't know the percentage of damaged buildings ($\pi_p$), PUMA's latent-state approach bypasses this requirement. This makes it significantly more practical for "unseen" disasters where such statistics are unavailable.
- Discriminative Power: As shown in the t-SNE visualizations (Figure 1), PUMA creates much tighter clusters and a clearer separation between classes compared to standard cross-entropy or other PU methods.
Key Results
- Superior Accuracy: PUMA consistently outperformed state-of-the-art PU learning and GFM fine-tuning methods across all benchmarks. For instance, on the tsunami dataset, it improved the F1-score by nearly 6 percentage points over the next best method.
- Extreme Label Efficiency: In the Los Angeles wildfire application, PUMA achieved an F1-score of 92.81% using only 50 damaged samples. To reach similar performance, standard supervised fine-tuning required 40 times more labels.
- Robustness: The framework proved compatible with various Parameter-Efficient Fine-Tuning (PEFT) strategies like LoRA and VPT, as well as different backbones (CLIP, RemoteCLIP, etc.), proving it isn't tied to one specific architecture.
Limitations and Future Work
While PUMA excels at binary classification (damaged vs. undamaged), the authors note that real-world damage assessment often requires more granular levels (e.g., minor damage, major damage, destroyed). Future work aims to extend the framework to multi-class damage assessment and explore its application in other weakly supervised remote sensing tasks, such as flood mapping or forest disturbance detection.
Summary updated