Spectrally Measured Edge Width for Supervising Water Segmentation

Water-segmentation networks trained on satellite imagery are taught that every pixel is either water or land, right up to the drawn shoreline. At 10 to 30 metre resolution that is false. A single shoreline pixel often mixes open water, wet mud and reeds, and how wide this mixed strip is changes along the shore with slope, turbidity and vegetation. Current boundary-aware methods respond in one of two ways. Some sharpen the predicted edge inside a fixed band two or three pixels wide. Others soften labels using geometry alone, grading them by distance from the edge. Neither uses the information that actually records how mixed each pixel is: its multi-band spectrum.

We propose to read that mixing directly from the spectrum and use it as the training target. Around each hand-drawn boundary, we estimate a local "pure water" spectrum and a local "pure land" spectrum. We then express each transition pixel as a linear mixture of the two, which gives a water fraction between 0 and 1. A per-location separability score, based on the water index contrast between the two pure spectra, then sets two things: how much to trust that fraction, and how wide a tolerance band the boundary should get. The network learns soft, physically grounded targets where the scene is mixed and hard targets elsewhere. The claim is that the gain comes from the spectral measurement itself and not from softening edges in general. We test that claim directly against geometric softening of matched width.

Core Claim

Spectral ratio indices are normally used to make the final water/land decision: threshold the Modified Normalized Difference Water Index (MNDWI, green minus shortwave infrared over their sum), or feed it as an extra input channel. We think their more useful role is as a measuring instrument for supervision. Within a small window around a labelled boundary, write $x$ for a pixel's reflectance vector across all Sentinel-2 bands. Write $e_W$ and $e_L$ for the mean spectra of pixels confidently inside and outside the hand label in that window. Under linear mixing, the water fraction is estimated as

$$\hat f(x) = \mathrm{clip}\!\left(\frac{(x - e_L)^\top (e_W - e_L)}{\lVert e_W - e_L \rVert^2},\,0,\,1\right).$$

Let $c$ be the absolute MNDWI difference between $e_W$ and $e_L$, normalized by the within-class spread of MNDWI in the window. We call $c$ the local separability. Transition pixels get target $\hat f$ with weight $\min(1, c/c_0)$, where $c_0$ is a single global constant. The width of the run of pixels with $\hat f$ strictly between 0.05 and 0.95, measured along the boundary normal, defines a local tolerance radius for a boundary loss.

Where water and land are spectrally distinct, for example clear lake against dry rock, this collapses to near-binary supervision with a narrow tolerance. Where they are not, for example turbid floodwater against wet soil, the target is soft, its weight is low, and the tolerance is wide. The network is never forced to put a sharp step where the physics does not support one.

This differs from the closest work in specific ways. Bounding Maps for Universal Lesion Detection showed in CT lesion detection that continuous targets, graded linearly from 1.0 at the centre to 0.5 at the boundary, beat binary supervision. Its grading is purely geometric, which makes it our main control rather than our method. IMOSWFM estimates sub-pixel water fractions from Zhuhai-1 hyperspectral imagery and reports them as the final product. We instead use a cheaper two-endmember, label-anchored unmixing only to build training targets for a segmentation network, and we sidestep its open problem of choosing endmembers by taking them from the hand label locally. Boundary Loss for Remote Sensing Imagery Semantic Segmentation introduced a differentiable boundary-F1 loss with a fixed 2–3 pixel band. Our tolerance varies per location and is measured from the spectrum.

Research Program

The work has three parts.

The first part is target construction for Sentinel-2. For every hand-labelled chip, we compute local endmembers in sliding windows, sized at around 15 pixels as a starting point; the size is tuned once on validation data. We then compute $\hat f$, the separability $c$ and the per-pixel tolerance radius. Windows without enough confident pixels on both sides of the boundary fall back to binary labels. We record how often that happens.

The second part is the training objective. Outside the transition zone we use standard binary cross-entropy plus an IoU loss. Inside it we use cross-entropy against $\hat f$, weighted by separability. On top of this we add the boundary-F1 surrogate of Bokhovkin and Burnaev, but with the fixed buffer replaced by the per-pixel tolerance radius. The objective is independent of architecture. We test it on a plain U-Net that takes all Sentinel-2 bands plus an MNDWI channel. We also test it on the BASNet encoder-decoder with residual refinement used by the Sen1Floods11 fusion study, to check that it adds to a strong boundary architecture rather than replacing one.

The third part is a ground-truthed fraction test. Sen1Floods11 has no sub-pixel ground truth, so we build some. We aggregate 10 m Sentinel-2 bands and their 10 m hand labels into 3×3 blocks, which mimics 30 m Landsat-like pixels. The mean of the fine labels in each block becomes a true coarse-pixel water fraction. We then train at the coarse resolution with each supervision scheme and measure how well each recovers the true fractions and the narrow channels that blocky binary labels erase.

Our target construction does not know the aggregation was done. A caveat applies: the true fractions come from 10 m binary labels, so the test checks consistency across scales rather than physical truth.

Baselines

The naive reference is Otsu thresholding of MNDWI on each chip, with no learning. On Sen1Floods11 the dataset authors report Otsu at mIoU 0.4571 on permanent water and an FCNN trained on Sentinel-2 weak labels at mIoU 0.3389 on flood water (Sen1Floods11). We rerun Otsu on our own splits so the comparison is exact. It shows how far simple spectral separation goes and where it breaks, at shadows and turbid water.

The strongest existing method is BASNet trained with focal, SSIM and IoU losses on fused Sentinel-1 and Sentinel-2 inputs. It reports mIoU of 52.99%, IoU of 52.30% and overall accuracy of 92.81% on the Sen1Floods11 test set, and mIoU of 47.88% on the held-out Bolivia set (the Sen1Floods11 fusion study). It is the strongest competitor because it already combines a boundary-refinement module with a loss that targets hard edge pixels on exactly this data.

We retrain it ourselves on identical splits, inputs and augmentation, with the same number of epochs and the same hyperparameter search budget, measured in trials, as our variant. We then add our objective to the same network. Beating it means two things. First, a gain in boundary F1 at 1 and 2 pixel tolerance and in IoU on narrow water bodies that exceeds seed-to-seed variation across five seeds. Second, no loss of more than one point in overall IoU, on both the main test set and Bolivia.

The fixed-width boundary baseline is the differentiable boundary-F1 loss, which on ISPRS Potsdam raised IoU from 92.56% with IoU loss alone to 93.85% (Boundary Loss for Remote Sensing Imagery Semantic Segmentation). We run it on the U-Net with buffer widths of 1, 2, 3 and 5 pixels and report its best width chosen on validation, so it gets every chance.

The ablation that isolates our key ingredient is geometric soft labels. These are targets graded by signed distance to the label edge, in the manner of Bounding Maps, using a per-chip width matched to the mean transition width our spectral method produces. They are also given the same varying tolerance radius, but derived from distance alone. A second ablation removes the separability weighting, so all spectral fractions are trusted equally. This tests whether knowing where the classes are inseparable matters.

Why not just…?

Why not just threshold MNDWI with Otsu or an adaptive threshold? Ratio indices separate clear water from dry land well, but they fail exactly where boundaries are ambiguous and where other surfaces mimic water. The Sen1Floods11 fusion study reports deep models beating Otsu by over 18 mIoU points. Random Forests beat MNDWI with Otsu across Central Asian Landsat data (Central Asia GEE study). We keep Otsu as the naive baseline, and we use the index only where it is informative: as a local, label-anchored measurement, not a global decision rule.

Why not just concatenate MNDWI as an input channel? This is a serious alternative, and it is why we did not propose injecting indices into the network's internal boundary-attention modules. A network given log-transformed bands can largely compute such ratios itself. So every model we train, baselines included, gets all bands plus MNDWI as input. Our claim concerns the training targets, not the inputs. If input concatenation alone closed the gap, we would expect the plain U-Net baseline to match our variant on narrow-water IoU and fraction error. The experiment measures this directly.

Why not just soften edge labels geometrically, with label smoothing or distance-graded maps? This is the main threat. Bounding Maps already shows that continuous grading helps in CT. If our gains come only from removing hard 0/1 steps at edges, distance-graded targets of matched width will match us. We expect them to fall short where the true transition width varies along a single shoreline, such as braided rivers and floodplains with emergent vegetation, and where low separability should lower the weight of a pixel. We test this by splitting results according to the within-chip variance of the estimated transition width. If the geometric ablation ties us everywhere, including the high-variance chips, the spectral component adds nothing and we will report that.

Why not just use an RGB-pretrained foundation model or morphological post-processing? Post-processing such as morphological filtering or a conditional random field (a probabilistic model that smooths labels toward image edges) cannot recover a narrow channel that pooling has already erased. It also cannot tell a shadow from water. The PraNet-variant work needed in-network reverse attention before its polarization post-processing helped (Parallel Res2Net with Reverse Attention). BASeg needed custom modules and losses on top of a frozen DINOv3 backbone to reach 68.86% mIoU on GCD-25k. Its angle-based loss assumes the straight lines of man-made objects, which do not fit shorelines. We include a conditional-random-field post-processing step on the U-Net baseline as a cheap control.

Proposed Experiments

The main benchmark is Sen1Floods11, using its hand-labelled Sentinel-2 chips for supervision and the official test and Bolivia splits for evaluation. We also run a variant that adds Sentinel-1 radar bands, to compare with the fused BASNet setting. We report IoU and mIoU so the numbers can be compared with published figures. We also report boundary F1 at 1, 2 and 3 pixel tolerance. We report IoU restricted to connected water components narrower than five pixels, and the false-positive rate inside shadow regions, identified with a low-NIR, low-SWIR, non-water mask. On the 30 m aggregation test, we report fraction RMSE and narrow-channel recall.

IMOSWFM's reported fraction RMSE of 0.2265 comes from different, hyperspectral data. It serves as context only, not as a comparable baseline. If the Sentinel-2 Qinghai–Tibet lake set used by ViTenc-UNet can be obtained, we will use it as a transfer test on alpine shorelines with mountain shadows. Its reported IoU of 98.68% suggests most gains there will appear only in boundary metrics.

Three checks decide whether the claim holds. First, spectral soft targets must beat matched-width geometric soft targets on fraction RMSE and on narrow-water IoU, with the advantage concentrated in chips where the transition width varies most. Second, removing the separability weight must increase errors in low-separability zones such as turbid floodwater and wet soil, which shows that measuring inseparability matters. Third, false positives from shadows must not rise relative to binary training. A soft target that leaks into shadowed areas would be a failure of the method, not a side effect to explain away.