Accuracy and efficiency of compact foundation models for surface-water mapping during floods

Parastoo Farajpoor, Mohammadreza NarimaniarXiv 2026 · arXiv

TL;DR

This paper evaluates the Prithvi-EO-2.0-Tiny geospatial foundation model for flood mapping, finding that while pretraining significantly improves transformer performance—especially with limited data—it doesn't necessarily outperform a well-tuned, task-specific U-Net. The real "win" for compact foundation models is computational efficiency, offering nearly 2x faster inference on GPUs and 2.5x on CPUs compared to similarly sized convolutional networks.

What Problem Does This Paper Address?

Rapidly mapping surface water during floods is critical for disaster response, but creating accurate maps from satellite imagery is challenging. Traditionally, researchers have used simple spectral indices (like NDWI) or task-specific deep learning models like U-Nets.

Recently, Geospatial Foundation Models (GFMs)—large models pretrained on massive unlabeled satellite datasets—have emerged as a promising alternative. The hope is that these models can achieve higher accuracy with fewer labeled examples ("label efficiency") and generalize better to new geographic regions. However, the authors argue that the community often lacks rigorous comparisons against strong baselines. They set out to answer: Does pretraining actually beat a simple U-Net? And is the extra complexity worth the cost?

Methods

The researchers compared several architectures using the Prithvi-EO-2.0 family (developed by NASA and IBM) as their primary test subject:

  • Prithvi-Tiny Variants: They tested a fine-tuned version (updating all weights), a frozen-encoder version (only updating the decoder), and a randomly initialized version (no pretraining).
  • Baselines: A size-matched U-Net (7.76M parameters), a larger Prithvi-100M model, and standard spectral indices (NDWI/MNDWI).
  • Evaluation Protocols:
    • Protocol A: A standard train/test split where the model sees some chips from an event during training and is tested on others from the same event.
  • Protocol B (LOEO): A "Leave-One-Event-Out" approach to test geographic transferability—evaluating the model on an entire flood event it has never seen before.

The workflow, including the architecture of the Prithvi segmentation network, is detailed in Figure 2:

Figure 2: Processing, architectures, and computational workflow. (A) Data preparation, evaluation allocations, methods, and diagnostics. The leave-one-event-out shorthand refers to exclusion from the fixed benchmark training/validation pools; global learning rates were selected once under Protocol A and reused. (B) Single-date six-band Prithvi segmentation. Tiny features are taken from blocks 3, 6, 9, and 12 and decoded through a feature pyramid and UPerNet head. The decoder family is shared, but the non-encoder parameter count differs: approximately 2.15 M for Tiny and 5.73 M for the larger model. (C) Local preparation, Colab training, and local analysis. The 106 reported training runs comprise 12 selection runs and 94 evaluation runs.

Datasets

The study utilized the Sen1Floods11 dataset, specifically the hand-labeled Sentinel-2 subset.

  • Scope: 446 image chips (512×512 pixels) covering 11 global flood events.
  • Geographic Diversity: Includes events in the USA, Spain, Pakistan, India, and a specific holdout evaluation set in Bolivia.
  • Input: Six multispectral bands (Blue, Green, Red, Narrow NIR, and two SWIR bands).

The geographic distribution and example chips are shown in Figure 1:

Figure 1: Geographic coverage and the Bolivia holdout. (A) Locations, image dates, and hand-labeled chip counts for the 11 Sen1Floods11 events; symbol area scales with chip count. Bolivia is the Protocol A event holdout. (B) Bolivia chip locations and the metadata event footprint over Copernicus GLO-90 relief, with Natural Earth boundaries and rivers. (C) Two example chips, their reference-water boundaries, and Copernicus GLO-30 relief; unlabeled pixels are faded. Panel A uses an Equal Earth projection; the regional panels show WGS 84 geographic coordinates. The source cartography uses vertical exaggeration of 3 for regional shading and 12 for local shading, not for quantitative terrain analysis. Footprints describe the supplied event/chip coverage, not independently measured flood extents.

Key Insights and Ideas

  • Pretraining is a "Warm Start," not a "Better Finish": Pretraining helped the Tiny transformer architecture significantly outperform its randomly initialized counterpart (a gain of ~0.046 IoU with very few labels). However, this wasn't enough to consistently beat the U-Net baseline.
  • The Efficiency Dividend: The most compelling reason to use compact GFMs like Prithvi-Tiny isn't necessarily accuracy, but speed. Because the Vision Transformer (ViT) architecture is more computationally efficient for inference than the U-Net's convolutions, it processed images much faster.
  • The "Boundary" Problem: Most errors occurred at the edges of water bodies. The authors suggest that the 16x16 pixel patch tokenization used in transformers might struggle with fine spatial details compared to the skip connections in a U-Net.
  • Event-Level Volatility: Performance varied wildly between different flood events. A model might perform excellently in Paraguay but struggle in Pakistan, highlighting that "global" accuracy scores can hide local failures.

Key Results

  • Accuracy: On the standard test set, U-Net achieved the highest IoU (0.856), followed by Prithvi-100M (0.841) and Prithvi-Tiny (0.833).
  • Label Efficiency: When training data was slashed to just 25 chips, Prithvi-Tiny maintained an IoU of 0.814, showing strong resilience, though U-Net still led at 0.831.
  • Inference Speed: Prithvi-Tiny was a clear winner here. It processed a 512x512 chip in 16.1 ms on a GPU (vs. 30.6 ms for U-Net) and 258 ms on a CPU (vs. 665 ms for U-Net).
  • Reliability: Using predictive entropy (a measure of model uncertainty) proved highly effective at flagging erroneous pixels, with an AUROC of 0.92–0.95.

Qualitative comparisons of these results across different events can be seen in Figure 4:

Figure 1: Geographic coverage and the Bolivia holdout. (A) Locations, image dates, and hand-labeled chip counts for the 11 Sen1Floods11 events; symbol area scales with chip count. Bolivia is the Protocol A event holdout. (B) Bolivia chip locations and the metadata event footprint over Copernicus GLO-90 relief, with Natural Earth boundaries and rivers. (C) Two example chips, their reference-water boundaries, and Copernicus GLO-30 relief; unlabeled pixels are faded. Panel A uses an Equal Earth projection; the regional panels show WGS 84 geographic coordinates. The source cartography uses vertical exaggeration of 3 for regional shading and 12 for local shading, not for quantitative terrain analysis. Footprints describe the supplied event/chip coverage, not independently measured flood extents.

Limitations and Future Work

  • Domain Mismatch: The Prithvi model was pretrained on surface reflectance data (HLS), but the test dataset used top-of-atmosphere (TOA) reflectance. While normalization helps, this mismatch likely hindered the foundation model's performance.
  • Spatial Resolution: The transformer's patch-based approach limits its ability to map narrow channels or complex shorelines. Future work could explore hybrid "U-Prithvi" architectures that combine pretrained encoders with convolutional skip connections.
  • Temporal Data: This study focused on single-date imagery. The authors note that incorporating time-series data (pre-flood vs. during-flood) is a logical next step to better distinguish permanent water from new flooding.

Summary updated

Abstract

Pretrained Earth observation models may reduce the effort needed to map surface water, but their value depends on the alternative against which they are compared. We evaluated Prithvi-EO-2.0-Tiny using the 446 hand-labeled Sentinel-2 chips from 11 flood events in Sen1Floods11. A fine-tuned model, a frozen-encoder model, and an identical randomly initialized architecture were compared with a similarly sized U-Net, a larger Prithvi model, and spectral-index thresholds. Experiments varied the number of training chips, excluded events from model fitting, and measured computational cost and prediction reliability. With 252 training chips, U-Net achieved a water intersection-over-union (IoU) of 0.856 on the official test set, compared with 0.833 for fine-tuned Prithvi-Tiny. Pretraining improved the tiny transformer over random initialization by a reported 0.046 IoU with 25 training chips and 0.023 with 252 chips, but did not reverse the U-Net ranking. Across 11 event-exclusion evaluations, mean IoU was 0.768 for fine-tuned Prithvi-Tiny and 0.777 for U-Net; the paired interval included zero. Fine-tuned Prithvi-Tiny processed a 512x512-pixel chip in 16.1 ms on a GPU and 258 ms on a laptop CPU, compared with 30.6 and 665 ms for U-Net. Errors were concentrated near water boundaries, and high predictive entropy identified many, but not all, erroneous pixels. These results support a specific role for compact pretrained models: lower measured computational demand with competitive accuracy, rather than consistent superiority. Event-dependent performance, shared hyperparameter selection, and the distinction between observed water and newly inundated land delimit the conclusions.
BibTeX
@misc{farajpoor2026accuracy,
  author = {Parastoo Farajpoor and Mohammadreza Narimani},
  title = {{Accuracy and efficiency of compact foundation models for surface-water mapping during floods}},
  year = {2026},
  eprint = {2610.34904},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.34904},
}
Figure